these 13 derivations will be built: /nix/store/85sd6x424d3d2ahcvrx4l6vmgb4dzd2h-nixbot-effects.drv /nix/store/qmi39i9h261xqaf6nmmdidnfxldw492w-unit-nixbot.service.drv /nix/store/yz8rn4nrr3km4caf9wdl038naj0kxjyy-system-units.drv /nix/store/rysf0mc1vs2kg7a32axibiz3yvcwg4sm-etc.drv /nix/store/ilw3fg5477d7nf6h78igk2advbsmw41j-activate.drv /nix/store/av3rmdz015iwh21hi87qn04m7f29196g-nixos-system-gitlab-test.drv /nix/store/ckc0xq5szg0n8r7xkpsk2yi2vy9x99k4-closure-info.drv /nix/store/khd5g91c6fm1mblwsskd8sarg6qdqvmv-closure-info.drv /nix/store/bnn036wv9idnab5rzz10dvjwmw7yhy3p-run-nixos-vm.drv /nix/store/kw6v63clymq15hjjhfyqzckdm173swyw-nixos-vm.drv /nix/store/pmpqrlz6nkf08whk79prf8h2lqm6zgn5-driverConfiguration.json.drv /nix/store/cjsmyh9nqz3k7dp9j1kyf6gfxjsvpijf-nixos-test-driver-nixbot-gitlab.drv /nix/store/qk680n52xkviw110r61s6s3d9sp38vfv-vm-test-run-nixbot-gitlab.drv building '/nix/store/85sd6x424d3d2ahcvrx4l6vmgb4dzd2h-nixbot-effects.drv' on 'ssh-ng://nix@jamie' building '/nix/store/85sd6x424d3d2ahcvrx4l6vmgb4dzd2h-nixbot-effects.drv' nixbot-effects> Sourcing python-remove-tests-dir-hook nixbot-effects> Sourcing python-catch-conflicts-hook.sh nixbot-effects> Sourcing python-remove-bin-bytecode-hook.sh nixbot-effects> Sourcing pypa-build-hook nixbot-effects> Using pypaBuildPhase nixbot-effects> Sourcing python-runtime-deps-check-hook nixbot-effects> Using pythonRuntimeDepsCheckHook nixbot-effects> Sourcing pypa-install-hook nixbot-effects> Using pypaInstallPhase nixbot-effects> Sourcing python-imports-check-hook.sh nixbot-effects> Using pythonImportsCheckPhase nixbot-effects> Sourcing python-namespaces-hook nixbot-effects> Sourcing python-catch-conflicts-hook.sh nixbot-effects> Running phase: unpackPhase nixbot-effects> unpacking source archive /nix/store/d39xb4grzwjgfvih4zz1khwi4cxqrn9d-nixbot_effects nixbot-effects> source root is nixbot_effects nixbot-effects> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot_effects/pyproject.toml" nixbot-effects> Running phase: patchPhase nixbot-effects> Running phase: updateAutotoolsGnuConfigScriptsPhase nixbot-effects> Running phase: configurePhase nixbot-effects> no configure script, doing nothing nixbot-effects> Running phase: buildPhase nixbot-effects> Executing pypaBuildPhase nixbot-effects> Creating a wheel... nixbot-effects> pypa build flags: --no-isolation --outdir dist/ --wheel nixbot-effects> * Getting build dependencies for wheel... nixbot-effects> * Building wheel... nixbot-effects> Successfully built nixbot_effects-0.0.1-py3-none-any.whl nixbot-effects> Finished creating a wheel... nixbot-effects> Finished executing pypaBuildPhase nixbot-effects> Running phase: pythonRuntimeDepsCheckHook nixbot-effects> Executing pythonRuntimeDepsCheck nixbot-effects> Checking runtime dependencies for nixbot_effects-0.0.1-py3-none-any.whl nixbot-effects> Finished executing pythonRuntimeDepsCheck nixbot-effects> Running phase: installPhase nixbot-effects> Executing pypaInstallPhase nixbot-effects> Successfully installed nixbot_effects-0.0.1-py3-none-any.whl nixbot-effects> Finished executing pypaInstallPhase nixbot-effects> Running phase: pythonOutputDistPhase nixbot-effects> Executing pythonOutputDistPhase nixbot-effects> Finished executing pythonOutputDistPhase nixbot-effects> Running phase: fixupPhase nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/qyicpbzcs9ps3izlaq7p2wfkw6jhd2wc-nixbot-effects nixbot-effects> checking for references to /build/ in /nix/store/qyicpbzcs9ps3izlaq7p2wfkw6jhd2wc-nixbot-effects... nixbot-effects> patching script interpreter paths in /nix/store/qyicpbzcs9ps3izlaq7p2wfkw6jhd2wc-nixbot-effects nixbot-effects> stripping (with command strip and flags -S -p) in /nix/store/qyicpbzcs9ps3izlaq7p2wfkw6jhd2wc-nixbot-effects/lib /nix/store/qyicpbzcs9ps3izlaq7p2wfkw6jhd2wc-nixbot-effects/bin nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/bj47r63iga31zggzf959m4r3m8pgyyaj-nixbot-effects-dist nixbot-effects> checking for references to /build/ in /nix/store/bj47r63iga31zggzf959m4r3m8pgyyaj-nixbot-effects-dist... nixbot-effects> patching script interpreter paths in /nix/store/bj47r63iga31zggzf959m4r3m8pgyyaj-nixbot-effects-dist nixbot-effects> Rewriting #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13 to #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13 nixbot-effects> wrapping `/nix/store/qyicpbzcs9ps3izlaq7p2wfkw6jhd2wc-nixbot-effects/bin/nixbot-effects'... nixbot-effects> Executing pythonRemoveTestsDir nixbot-effects> Finished executing pythonRemoveTestsDir nixbot-effects> Running phase: installCheckPhase nixbot-effects> no Makefile or custom installCheckPhase, doing nothing nixbot-effects> Running phase: pythonCatchConflictsPhase nixbot-effects> Running phase: pythonRemoveBinBytecodePhase nixbot-effects> Running phase: pythonImportsCheckPhase nixbot-effects> Executing pythonImportsCheckPhase building '/nix/store/qmi39i9h261xqaf6nmmdidnfxldw492w-unit-nixbot.service.drv' building '/nix/store/yz8rn4nrr3km4caf9wdl038naj0kxjyy-system-units.drv' building '/nix/store/rysf0mc1vs2kg7a32axibiz3yvcwg4sm-etc.drv' building '/nix/store/ilw3fg5477d7nf6h78igk2advbsmw41j-activate.drv' building '/nix/store/av3rmdz015iwh21hi87qn04m7f29196g-nixos-system-gitlab-test.drv' nixos-system-gitlab-test> structuredAttrs is enabled building '/nix/store/ckc0xq5szg0n8r7xkpsk2yi2vy9x99k4-closure-info.drv' closure-info> structuredAttrs is enabled building '/nix/store/khd5g91c6fm1mblwsskd8sarg6qdqvmv-closure-info.drv' closure-info> structuredAttrs is enabled building '/nix/store/bnn036wv9idnab5rzz10dvjwmw7yhy3p-run-nixos-vm.drv' building '/nix/store/kw6v63clymq15hjjhfyqzckdm173swyw-nixos-vm.drv' building '/nix/store/pmpqrlz6nkf08whk79prf8h2lqm6zgn5-driverConfiguration.json.drv' driverConfiguration.json> structuredAttrs is enabled building '/nix/store/cjsmyh9nqz3k7dp9j1kyf6gfxjsvpijf-nixos-test-driver-nixbot-gitlab.drv' nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-nixbot-gitlab> All checks passed! nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint nixos-test-driver-nixbot-gitlab> All checks passed! building '/nix/store/qk680n52xkviw110r61s6s3d9sp38vfv-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie' building '/nix/store/qk680n52xkviw110r61s6s3d9sp38vfv-vm-test-run-nixbot-gitlab.drv' vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state vm-test-run-nixbot-gitlab> start all VLans vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds) vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds vm-test-run-nixbot-gitlab> run the VM test script vm-test-run-nixbot-gitlab> additionally exposed symbols: vm-test-run-nixbot-gitlab> gitlab, vm-test-run-nixbot-gitlab> vlan1, vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh vm-test-run-nixbot-gitlab> start all VMs vm-test-run-nixbot-gitlab> gitlab: starting vm vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13) vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image... vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.4XKRO5MJzb', fmt=raw size=1073741824 vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025) vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done vm-test-run-nixbot-gitlab> (finished: start all VMs, in 37.15 seconds) vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: b090a739-8458-4c6d-9ecc-d66162d499e6 vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks: vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376 vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created. vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image... vm-test-run-nixbot-gitlab> gitlab # Created Nix store image. vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Booting from ROM... vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vg6nqmxghq3ys5636j4rn33azchwqhfb-nixos-system-gitlab-test/init regInfo=/nix/store/6dlvm7dp7q6vy5rsfw8vmwaf41l3g1wa-closure-info/registration console=ttyS0,115200n8 console=tty0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-provided physical RAM map: vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NX (Execute Disable) protection: active vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] APIC: Static calls initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SMBIOS 2.8 present. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Hypervisor detected: KVM vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] kvm-clock: using sched offset of 508516873 cycles vm-test-run-nixbot-gitlab> gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor vm-test-run-nixbot-gitlab> gitlab # [ 0.000806] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs vm-test-run-nixbot-gitlab> gitlab # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT vm-test-run-nixbot-gitlab> gitlab # [ 0.000897] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.002757] found SMP MP-table at [mem 0x000f5470-0x000f547f] vm-test-run-nixbot-gitlab> gitlab # [ 0.002768] Using GB pages for direct mapping vm-test-run-nixbot-gitlab> gitlab # [ 0.002894] RAMDISK: [mem 0xbe4e0000-0xbffcffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.002901] ACPI: Early table checksum verification disabled vm-test-run-nixbot-gitlab> gitlab # [ 0.002904] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) vm-test-run-nixbot-gitlab> gitlab # [ 0.002908] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002912] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002919] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002921] ACPI: FACS 0x00000000BFFE0000 000040 vm-test-run-nixbot-gitlab> gitlab # [ 0.002923] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002924] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002926] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002927] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] vm-test-run-nixbot-gitlab> gitlab # [ 0.002928] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] vm-test-run-nixbot-gitlab> gitlab # [ 0.002929] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] vm-test-run-nixbot-gitlab> gitlab # [ 0.002929] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] vm-test-run-nixbot-gitlab> gitlab # [ 0.002930] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] vm-test-run-nixbot-gitlab> gitlab # [ 0.002930] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] vm-test-run-nixbot-gitlab> gitlab # [ 0.003420] No NUMA configuration found vm-test-run-nixbot-gitlab> gitlab # [ 0.003421] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.003424] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005959] Zone ranges: vm-test-run-nixbot-gitlab> gitlab # [ 0.005959] DMA [mem 0x0000000000001000-0x0000000000ffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005961] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005962] Normal [mem 0x0000000100000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005963] Device empty vm-test-run-nixbot-gitlab> gitlab # [ 0.005964] Movable zone start for each node vm-test-run-nixbot-gitlab> gitlab # [ 0.005965] Early memory node ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.005965] node 0: [mem 0x0000000000001000-0x000000000009efff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005966] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005967] node 0: [mem 0x0000000100000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005968] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005987] On node 0, zone DMA: 1 pages in unavailable ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.006257] On node 0, zone DMA: 97 pages in unavailable ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.115909] On node 0, zone Normal: 40 pages in unavailable ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.116944] ACPI: PM-Timer IO Port: 0x608 vm-test-run-nixbot-gitlab> gitlab # [ 0.116962] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) vm-test-run-nixbot-gitlab> gitlab # [ 0.117010] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 vm-test-run-nixbot-gitlab> gitlab # [ 0.117016] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) vm-test-run-nixbot-gitlab> gitlab # [ 0.117020] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.117022] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.117023] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.117024] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.117026] ACPI: Using ACPI (MADT) for SMP configuration information vm-test-run-nixbot-gitlab> gitlab # [ 0.117028] ACPI: HPET id: 0x8086a201 base: 0xfed00000 vm-test-run-nixbot-gitlab> gitlab # [ 0.117034] TSC deadline timer available vm-test-run-nixbot-gitlab> gitlab # [ 0.117038] CPU topo: Max. logical packages: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.117039] CPU topo: Max. logical dies: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.117040] CPU topo: Max. dies per package: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.117043] CPU topo: Max. threads per core: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.117044] CPU topo: Num. cores per package: 4 vm-test-run-nixbot-gitlab> gitlab # [ 0.117044] CPU topo: Num. threads per package: 4 vm-test-run-nixbot-gitlab> gitlab # [ 0.117044] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.117070] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() vm-test-run-nixbot-gitlab> gitlab # [ 0.117090] kvm-guest: KVM setup pv remote TLB flush vm-test-run-nixbot-gitlab> gitlab # [ 0.117093] kvm-guest: setup PV sched yield vm-test-run-nixbot-gitlab> gitlab # [ 0.117121] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.117123] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.117124] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.117126] [mem 0xc0000000-0xfeffbfff] available for PCI devices vm-test-run-nixbot-gitlab> gitlab # [ 0.117127] Booting paravirtualized kernel on KVM vm-test-run-nixbot-gitlab> gitlab # [ 0.117131] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.121690] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 vm-test-run-nixbot-gitlab> gitlab # [ 0.123899] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 vm-test-run-nixbot-gitlab> gitlab # [ 0.123952] kvm-guest: PV spinlocks enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.123954] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.123957] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vg6nqmxghq3ys5636j4rn33azchwqhfb-nixos-system-gitlab-test/init regInfo=/nix/store/6dlvm7dp7q6vy5rsfw8vmwaf41l3g1wa-closure-info/registration console=ttyS0,115200n8 console=tty0 vm-test-run-nixbot-gitlab> gitlab # [ 0.124064] Unknown kernel command line parameters "regInfo=/nix/store/6dlvm7dp7q6vy5rsfw8vmwaf41l3g1wa-closure-info/registration", will be passed to user space. vm-test-run-nixbot-gitlab> gitlab # [ 0.124079] random: crng init done vm-test-run-nixbot-gitlab> gitlab # [ 0.124080] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes vm-test-run-nixbot-gitlab> gitlab # [ 0.132771] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.137159] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.137237] software IO TLB: area num 4. vm-test-run-nixbot-gitlab> gitlab # [ 0.210186] Fallback order for Node 0: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.210195] Built 1 zonelists, mobility grouping on. Total pages: 1572726 vm-test-run-nixbot-gitlab> gitlab # [ 0.210197] Policy zone: Normal vm-test-run-nixbot-gitlab> gitlab # [ 0.212715] mem auto-init: stack:all(zero), heap alloc:on, heap free:off vm-test-run-nixbot-gitlab> gitlab # [ 0.222473] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.235791] allocated 12582912 bytes of page_ext vm-test-run-nixbot-gitlab> gitlab # [ 0.245613] ftrace: allocating 48565 entries in 192 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.245615] ftrace: allocated 192 pages with 2 groups vm-test-run-nixbot-gitlab> gitlab # [ 0.246491] Dynamic Preempt: lazy vm-test-run-nixbot-gitlab> gitlab # [ 0.246661] rcu: Preemptible hierarchical RCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.246661] rcu: RCU event tracing is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.246662] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.246664] Trampoline variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.246664] Rude variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.246664] Tracing variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.246665] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. vm-test-run-nixbot-gitlab> gitlab # [ 0.246666] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 vm-test-run-nixbot-gitlab> gitlab # [ 0.246680] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.246682] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.246683] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.251011] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 vm-test-run-nixbot-gitlab> gitlab # [ 0.251301] rcu: srcu_init: Setting srcu_struct sizes based on contention. vm-test-run-nixbot-gitlab> gitlab # [ 0.251421] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) vm-test-run-nixbot-gitlab> gitlab # [ 0.258736] Console: colour VGA+ 80x25 vm-test-run-nixbot-gitlab> gitlab # [ 0.258740] printk: legacy console [tty0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.302703] printk: legacy console [ttyS0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.501848] ACPI: Core revision 20250807 vm-test-run-nixbot-gitlab> gitlab # [ 0.503471] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.506360] APIC: Switch to symmetric I/O mode setup vm-test-run-nixbot-gitlab> gitlab # [ 0.508113] x2apic enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.509361] APIC: Switched APIC routing to: physical x2apic vm-test-run-nixbot-gitlab> gitlab # [ 0.511063] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() vm-test-run-nixbot-gitlab> gitlab # [ 0.513282] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() vm-test-run-nixbot-gitlab> gitlab # [ 0.516059] kvm-guest: setup PV IPIs vm-test-run-nixbot-gitlab> gitlab # [ 0.518378] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 vm-test-run-nixbot-gitlab> gitlab # [ 0.522582] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.525777] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010) vm-test-run-nixbot-gitlab> gitlab # [ 0.527818] x86/cpu: User Mode Instruction Prevention (UMIP) activated vm-test-run-nixbot-gitlab> gitlab # [ 0.529925] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 vm-test-run-nixbot-gitlab> gitlab # [ 0.530774] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.532778] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto vm-test-run-nixbot-gitlab> gitlab # [ 0.534775] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl vm-test-run-nixbot-gitlab> gitlab # [ 0.535775] Transient Scheduler Attacks: Mitigation: Clear CPU buffers vm-test-run-nixbot-gitlab> gitlab # [ 0.537774] Spectre V2 : Mitigation: Enhanced / Automatic IBRS vm-test-run-nixbot-gitlab> gitlab # [ 0.539774] Speculative Return Stack Overflow: Mitigation: Safe RET vm-test-run-nixbot-gitlab> gitlab # [ 0.540774] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization vm-test-run-nixbot-gitlab> gitlab # [ 0.541780] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier vm-test-run-nixbot-gitlab> gitlab # [ 0.542775] active return thunk: srso_alias_return_thunk vm-test-run-nixbot-gitlab> gitlab # [ 0.543803] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.544774] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.545774] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.546774] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' vm-test-run-nixbot-gitlab> gitlab # [ 0.547774] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' vm-test-run-nixbot-gitlab> gitlab # [ 0.548774] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' vm-test-run-nixbot-gitlab> gitlab # [ 0.549774] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.550774] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.551775] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)' vm-test-run-nixbot-gitlab> gitlab # [ 0.553775] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 vm-test-run-nixbot-gitlab> gitlab # [ 0.554774] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 vm-test-run-nixbot-gitlab> gitlab # [ 0.555774] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 vm-test-run-nixbot-gitlab> gitlab # [ 0.556774] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 vm-test-run-nixbot-gitlab> gitlab # [ 0.558774] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 vm-test-run-nixbot-gitlab> gitlab # [ 0.560774] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16 vm-test-run-nixbot-gitlab> gitlab # [ 0.561774] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24 vm-test-run-nixbot-gitlab> gitlab # [ 0.562774] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format. vm-test-run-nixbot-gitlab> gitlab # [ 0.592031] Freeing SMP alternatives memory: 44K vm-test-run-nixbot-gitlab> gitlab # [ 0.592777] pid_max: default: 32768 minimum: 301 vm-test-run-nixbot-gitlab> gitlab # [ 0.594881] LSM: initializing lsm=capability,landlock,yama,bpf,ima vm-test-run-nixbot-gitlab> gitlab # [ 0.595887] landlock: Up and running. vm-test-run-nixbot-gitlab> gitlab # [ 0.596774] Yama: becoming mindful. vm-test-run-nixbot-gitlab> gitlab # [ 0.597991] LSM support for eBPF active vm-test-run-nixbot-gitlab> gitlab # [ 0.600010] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.600929] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.604022] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) vm-test-run-nixbot-gitlab> gitlab # [ 0.605359] Performance Events: Fam17h+ core perfctr, AMD PMU driver. vm-test-run-nixbot-gitlab> gitlab # [ 0.605785] ... version: 2 vm-test-run-nixbot-gitlab> gitlab # [ 0.606776] ... bit width: 48 vm-test-run-nixbot-gitlab> gitlab # [ 0.607776] ... generic counters: 6 vm-test-run-nixbot-gitlab> gitlab # [ 0.608776] ... generic bitmap: 000000000000003f vm-test-run-nixbot-gitlab> gitlab # [ 0.609776] ... fixed-purpose counters: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.610776] ... fixed-purpose bitmap: 0000000000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.611776] ... value mask: 0000ffffffffffff vm-test-run-nixbot-gitlab> gitlab # [ 0.612776] ... max period: 00007fffffffffff vm-test-run-nixbot-gitlab> gitlab # [ 0.613776] ... global_ctrl mask: 000000000000003f vm-test-run-nixbot-gitlab> gitlab # [ 0.614902] signal: max sigframe size: 3376 vm-test-run-nixbot-gitlab> gitlab # [ 0.615870] rcu: Hierarchical SRCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.616780] rcu: Max phase no-delay instances is 400. vm-test-run-nixbot-gitlab> gitlab # [ 0.617969] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level vm-test-run-nixbot-gitlab> gitlab # [ 0.623319] smp: Bringing up secondary CPUs ... vm-test-run-nixbot-gitlab> gitlab # [ 0.624192] smpboot: x86: Booting SMP configuration: vm-test-run-nixbot-gitlab> gitlab # [ 0.624780] .... node #0, CPUs: #1 #2 #3 vm-test-run-nixbot-gitlab> gitlab # [ 0.628885] smp: Brought up 1 node, 4 CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.631779] smpboot: Total of 4 processors activated (19200.08 BogoMIPS) vm-test-run-nixbot-gitlab> gitlab # [ 0.633383] Memory: 6013396K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258300K reserved, 0K cma-reserved) vm-test-run-nixbot-gitlab> gitlab # [ 0.635775] devtmpfs: initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.636977] x86/mm: Memory block size: 128MB vm-test-run-nixbot-gitlab> gitlab # [ 0.640834] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.641827] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.642866] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). vm-test-run-nixbot-gitlab> gitlab # [ 0.643919] pinctrl core: initialized pinctrl subsystem vm-test-run-nixbot-gitlab> gitlab # [ 0.645104] PM: RTC time: 22:04:57, date: 2026-06-29 vm-test-run-nixbot-gitlab> gitlab # [ 0.648892] NET: Registered PF_NETLINK/PF_ROUTE protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.651125] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.651849] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.653845] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.654789] audit: initializing netlink subsys (disabled) vm-test-run-nixbot-gitlab> gitlab # [ 0.655981] audit: type=2000 audit(1782770697.389:1): state=initialized audit_enabled=0 res=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.656127] thermal_sys: Registered thermal governor 'fair_share' vm-test-run-nixbot-gitlab> gitlab # [ 0.657777] thermal_sys: Registered thermal governor 'bang_bang' vm-test-run-nixbot-gitlab> gitlab # [ 0.658776] thermal_sys: Registered thermal governor 'step_wise' vm-test-run-nixbot-gitlab> gitlab # [ 0.659776] thermal_sys: Registered thermal governor 'user_space' vm-test-run-nixbot-gitlab> gitlab # [ 0.660776] thermal_sys: Registered thermal governor 'power_allocator' vm-test-run-nixbot-gitlab> gitlab # [ 0.661828] cpuidle: using governor menu vm-test-run-nixbot-gitlab> gitlab # [ 0.665038] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 vm-test-run-nixbot-gitlab> gitlab # [ 0.666096] PCI: Using configuration type 1 for base access vm-test-run-nixbot-gitlab> gitlab # [ 0.666777] PCI: Using configuration type 1 for extended access vm-test-run-nixbot-gitlab> gitlab # [ 0.667998] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. vm-test-run-nixbot-gitlab> gitlab # [ 0.670833] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.671777] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.672777] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.673776] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.678401] ACPI: Added _OSI(Module Device) vm-test-run-nixbot-gitlab> gitlab # [ 0.681785] ACPI: Added _OSI(Processor Device) vm-test-run-nixbot-gitlab> gitlab # [ 0.685788] ACPI: Added _OSI(Processor Aggregator Device) vm-test-run-nixbot-gitlab> gitlab # [ 0.694824] ACPI: 1 ACPI AML tables successfully acquired and loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.703413] ACPI: Interpreter enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.706998] ACPI: PM: (supports S0 S3 S4 S5) vm-test-run-nixbot-gitlab> gitlab # [ 0.710792] ACPI: Using IOAPIC for interrupt routing vm-test-run-nixbot-gitlab> gitlab # [ 0.714128] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug vm-test-run-nixbot-gitlab> gitlab # [ 0.720784] PCI: Using E820 reservations for host bridge windows vm-test-run-nixbot-gitlab> gitlab # [ 0.726483] ACPI: Enabled 2 GPEs in block 00 to 0F vm-test-run-nixbot-gitlab> gitlab # [ 0.741697] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) vm-test-run-nixbot-gitlab> gitlab # [ 0.745794] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] vm-test-run-nixbot-gitlab> gitlab # [ 0.751833] acpiphp: Slot [3] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.753868] acpiphp: Slot [4] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.756867] acpiphp: Slot [5] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.758893] acpiphp: Slot [6] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.760819] acpiphp: Slot [7] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.762820] acpiphp: Slot [8] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.763818] acpiphp: Slot [9] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.764837] acpiphp: Slot [10] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.766818] acpiphp: Slot [11] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.767818] acpiphp: Slot [12] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.768838] acpiphp: Slot [13] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.769811] acpiphp: Slot [14] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.772090] acpiphp: Slot [15] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.772817] acpiphp: Slot [16] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.775871] acpiphp: Slot [17] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.778903] acpiphp: Slot [18] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.781869] acpiphp: Slot [19] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.784869] acpiphp: Slot [20] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.788868] acpiphp: Slot [21] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.791906] acpiphp: Slot [22] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.794888] acpiphp: Slot [23] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.797867] acpiphp: Slot [24] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.800867] acpiphp: Slot [25] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.804906] acpiphp: Slot [26] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.807876] acpiphp: Slot [27] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.811945] acpiphp: Slot [28] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.813851] acpiphp: Slot [29] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.816879] acpiphp: Slot [30] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.818849] acpiphp: Slot [31] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.821969] PCI host bridge to bus 0000:00 vm-test-run-nixbot-gitlab> gitlab # [ 0.824822] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] vm-test-run-nixbot-gitlab> gitlab # [ 0.828784] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.831778] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.834783] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.836778] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.838778] pci_bus 0000:00: root bus resource [bus 00-ff] vm-test-run-nixbot-gitlab> gitlab # [ 0.841683] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.850136] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.858176] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.869948] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] vm-test-run-nixbot-gitlab> gitlab # [ 0.873948] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.879785] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.883785] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.888785] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.894565] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.901912] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] vm-test-run-nixbot-gitlab> gitlab # [ 0.908013] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.910978] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI vm-test-run-nixbot-gitlab> gitlab # [ 0.916822] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB vm-test-run-nixbot-gitlab> gitlab # [ 0.921949] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x180 took 10742 usecs vm-test-run-nixbot-gitlab> gitlab # [ 0.927872] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.948925] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.951862] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.955957] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.959024] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.972782] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 13671 usecs vm-test-run-nixbot-gitlab> gitlab # [ 0.976955] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.988824] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] vm-test-run-nixbot-gitlab> gitlab # [ 0.992824] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.996917] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.002824] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.014028] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.026813] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] vm-test-run-nixbot-gitlab> gitlab # [ 1.029811] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.033888] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.043083] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.056820] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] vm-test-run-nixbot-gitlab> gitlab # [ 1.060835] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.065918] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.076100] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.089816] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] vm-test-run-nixbot-gitlab> gitlab # [ 1.092810] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.096888] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.106144] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.114821] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] vm-test-run-nixbot-gitlab> gitlab # [ 1.116793] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.117830] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.123482] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.128817] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] vm-test-run-nixbot-gitlab> gitlab # [ 1.129791] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.131831] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.136829] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.141807] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] vm-test-run-nixbot-gitlab> gitlab # [ 1.143792] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.145830] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.147791] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.151817] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.158833] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.159830] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.164842] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.170829] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] vm-test-run-nixbot-gitlab> gitlab # [ 1.172791] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.174859] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.181353] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.191812] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] vm-test-run-nixbot-gitlab> gitlab # [ 1.194812] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] vm-test-run-nixbot-gitlab> gitlab # [ 1.198887] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.222351] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.226201] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.230251] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.234181] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.235890] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 vm-test-run-nixbot-gitlab> gitlab # [ 1.240139] iommu: Default domain type: Translated vm-test-run-nixbot-gitlab> gitlab # [ 1.241776] iommu: DMA domain TLB invalidation policy: lazy mode vm-test-run-nixbot-gitlab> gitlab # [ 1.245098] ACPI: bus type USB registered vm-test-run-nixbot-gitlab> gitlab # [ 1.249176] usbcore: registered new interface driver usbfs vm-test-run-nixbot-gitlab> gitlab # [ 1.252895] usbcore: registered new interface driver hub vm-test-run-nixbot-gitlab> gitlab # [ 1.257886] usbcore: registered new device driver usb vm-test-run-nixbot-gitlab> gitlab # [ 1.265330] NetLabel: Initializing vm-test-run-nixbot-gitlab> gitlab # [ 1.267786] NetLabel: domain hash size = 128 vm-test-run-nixbot-gitlab> gitlab # [ 1.271787] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO vm-test-run-nixbot-gitlab> gitlab # [ 1.275875] NetLabel: unlabeled traffic allowed by default vm-test-run-nixbot-gitlab> gitlab # [ 1.279815] PCI: Using ACPI for IRQ routing vm-test-run-nixbot-gitlab> gitlab # [ 1.283946] pci 0000:00:02.0: vgaarb: setting as boot VGA device vm-test-run-nixbot-gitlab> gitlab # [ 1.284771] pci 0000:00:02.0: vgaarb: bridge control possible vm-test-run-nixbot-gitlab> gitlab # [ 1.284771] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none vm-test-run-nixbot-gitlab> gitlab # [ 1.295784] vgaarb: loaded vm-test-run-nixbot-gitlab> gitlab # [ 1.297041] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 vm-test-run-nixbot-gitlab> gitlab # [ 1.298777] hpet0: 3 comparators, 64-bit 100.000000 MHz counter vm-test-run-nixbot-gitlab> gitlab # [ 1.305894] clocksource: Switched to clocksource kvm-clock vm-test-run-nixbot-gitlab> gitlab # [ 1.311059] VFS: Disk quotas dquot_6.6.0 vm-test-run-nixbot-gitlab> gitlab # [ 1.312446] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) vm-test-run-nixbot-gitlab> gitlab # [ 1.315877] pnp: PnP ACPI init vm-test-run-nixbot-gitlab> gitlab # [ 1.320208] pnp: PnP ACPI: found 6 devices vm-test-run-nixbot-gitlab> gitlab # [ 1.336745] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns vm-test-run-nixbot-gitlab> gitlab # [ 1.343364] clocksource: Switched to clocksource acpi_pm vm-test-run-nixbot-gitlab> gitlab # [ 1.347924] NET: Registered PF_INET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.353629] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.378999] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.381677] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.384138] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.387850] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.390150] TCP: Hash tables configured (established 65536 bind 65536) vm-test-run-nixbot-gitlab> gitlab # [ 1.392760] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.395315] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.397744] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.400068] NET: Registered PF_UNIX/PF_LOCAL protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.401813] NET: Registered PF_XDP protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.403377] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] vm-test-run-nixbot-gitlab> gitlab # [ 1.405299] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.407219] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.409304] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.411399] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.413808] pci 0000:00:01.0: PIIX3: Enabling Passive Release vm-test-run-nixbot-gitlab> gitlab # [ 1.415608] pci 0000:00:00.0: Limiting direct PCI/PCI transfers vm-test-run-nixbot-gitlab> gitlab # [ 1.419061] ACPI: \_SB_.LNKD: Enabled at IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.422524] PCI: CLS 0 bytes, default 64 vm-test-run-nixbot-gitlab> gitlab # [ 1.423932] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) vm-test-run-nixbot-gitlab> gitlab # [ 1.424675] Trying to unpack rootfs image as initramfs... vm-test-run-nixbot-gitlab> gitlab # [ 1.425866] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB) vm-test-run-nixbot-gitlab> gitlab # [ 1.426019] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns vm-test-run-nixbot-gitlab> gitlab # [ 1.453258] Initialise system trusted keyrings vm-test-run-nixbot-gitlab> gitlab # [ 1.455621] workingset: timestamp_bits=40 max_order=21 bucket_order=0 vm-test-run-nixbot-gitlab> gitlab # [ 1.490336] Key type asymmetric registered vm-test-run-nixbot-gitlab> gitlab # [ 1.493688] Asymmetric key parser 'x509' registered vm-test-run-nixbot-gitlab> gitlab # [ 1.497756] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) vm-test-run-nixbot-gitlab> gitlab # [ 1.504201] io scheduler mq-deadline registered vm-test-run-nixbot-gitlab> gitlab # [ 1.507858] io scheduler kyber registered vm-test-run-nixbot-gitlab> gitlab # [ 1.518193] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled vm-test-run-nixbot-gitlab> gitlab # [ 1.524040] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A vm-test-run-nixbot-gitlab> gitlab # [ 1.534223] Linux agpgart interface v0.103 vm-test-run-nixbot-gitlab> gitlab # [ 1.537785] ACPI: bus type drm_connector registered vm-test-run-nixbot-gitlab> gitlab # [ 1.545409] usbcore: registered new interface driver usbserial_generic vm-test-run-nixbot-gitlab> gitlab # [ 1.550617] usbserial: USB Serial support registered for generic vm-test-run-nixbot-gitlab> gitlab # [ 1.555241] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. vm-test-run-nixbot-gitlab> gitlab # [ 1.555241] Please enable it if your BIOS has the CPPC option. vm-test-run-nixbot-gitlab> gitlab # [ 1.565322] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled vm-test-run-nixbot-gitlab> gitlab # [ 1.571383] drop_monitor: Initializing network drop monitor service vm-test-run-nixbot-gitlab> gitlab # [ 1.576660] NET: Registered PF_INET6 protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.582208] Segment Routing with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 1.584735] In-situ OAM (IOAM) with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 1.590876] IPI shorthand broadcast: enabled vm-test-run-nixbot-gitlab> gitlab # [ 1.604223] sched_clock: Marking stable (1340076858, 263460992)->(1687817884, -84280034) vm-test-run-nixbot-gitlab> gitlab # [ 1.611635] registered taskstats version 1 vm-test-run-nixbot-gitlab> gitlab # [ 1.616073] Loading compiled-in X.509 certificates vm-test-run-nixbot-gitlab> gitlab # [ 1.638381] Demotion targets for Node 0: null vm-test-run-nixbot-gitlab> gitlab # [ 1.640166] Key type .fscrypt registered vm-test-run-nixbot-gitlab> gitlab # [ 1.641447] Key type fscrypt-provisioning registered vm-test-run-nixbot-gitlab> gitlab # [ 1.643833] ima: No TPM chip found, activating TPM-bypass! vm-test-run-nixbot-gitlab> gitlab # [ 1.645548] ima: Allocated hash algorithm: sha1 vm-test-run-nixbot-gitlab> gitlab # [ 1.647037] ima: No architecture policies found vm-test-run-nixbot-gitlab> gitlab # [ 1.649032] PM: Magic number: 14:29:99 vm-test-run-nixbot-gitlab> gitlab # [ 1.651369] RAS: Correctable Errors collector initialized. vm-test-run-nixbot-gitlab> gitlab # [ 1.656814] clk: Disabling unused clocks vm-test-run-nixbot-gitlab> gitlab # [ 1.658131] PM: genpd: Disabling unused power domains vm-test-run-nixbot-gitlab> gitlab # [ 1.709147] Freeing initrd memory: 27584K vm-test-run-nixbot-gitlab> gitlab # [ 1.717615] Freeing unused decrypted memory: 2028K vm-test-run-nixbot-gitlab> gitlab # [ 1.726274] Freeing unused kernel image (initmem) memory: 3644K vm-test-run-nixbot-gitlab> gitlab # [ 1.731092] Write protecting the kernel read-only data: 32768k vm-test-run-nixbot-gitlab> gitlab # [ 1.738272] Freeing unused kernel image (text/rodata gap) memory: 1332K vm-test-run-nixbot-gitlab> gitlab # [ 1.743379] Freeing unused kernel image (rodata/data gap) memory: 816K vm-test-run-nixbot-gitlab> gitlab # [ 1.800296] x86/mm: Checked W+X mappings: passed, no W+X pages found. vm-test-run-nixbot-gitlab> gitlab # [ 1.802407] Run /init as init process vm-test-run-nixbot-gitlab> gitlab # [ 1.818082] systemd[1]: Inserted module 'autofs4' vm-test-run-nixbot-gitlab> gitlab # [ 1.847622] fuse: init (API version 7.45) vm-test-run-nixbot-gitlab> gitlab # [ 1.858300] ACPI: \_SB_.LNKC: Enabled at IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.868270] ACPI: \_SB_.LNKA: Enabled at IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.886120] ACPI: \_SB_.LNKB: Enabled at IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.939937] systemd[1]: Successfully made /usr/ read-only. vm-test-run-nixbot-gitlab> gitlab # [ 2.280930] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 2.302421] systemd[1]: Detected virtualization kvm. vm-test-run-nixbot-gitlab> gitlab # [ 2.306390] systemd[1]: Detected architecture x86-64. vm-test-run-nixbot-gitlab> gitlab # [ 2.310451] systemd[1]: Running in initrd. vm-test-run-nixbot-gitlab> gitlab # [ 2.315004] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 2.320236] systemd[1]: Hostname set to . vm-test-run-nixbot-gitlab> gitlab # [ 2.397016] systemd[1]: Queued start job for default target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 2.574606] systemd[1]: Created slice Slice /system/modprobe. vm-test-run-nixbot-gitlab> gitlab # [ 2.579805] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 2.586006] systemd[1]: Expecting device /dev/disk/by-label/nix-store... vm-test-run-nixbot-gitlab> gitlab # [ 2.591162] systemd[1]: Expecting device /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 2.596211] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.600238] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.604314] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 2.608028] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.612448] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 2.616992] systemd[1]: Listening on Journal Socket (/dev/log). vm-test-run-nixbot-gitlab> gitlab # [ 2.621093] systemd[1]: Listening on Journal Sockets. vm-test-run-nixbot-gitlab> gitlab # [ 2.624761] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 2.628315] systemd[1]: Listening on udev Kernel Socket. vm-test-run-nixbot-gitlab> gitlab # [ 2.630568] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.636411] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 2.651183] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 2.657109] systemd[1]: Starting Load Kernel Module configfs... vm-test-run-nixbot-gitlab> gitlab # [ 2.672359] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 2.682341] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 2.686702] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 2.690437] systemd-journald[151]: Collecting audit messages is disabled. vm-test-run-nixbot-gitlab> gitlab # [ 2.704768] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 2.732619] netfs: FS-Cache loaded vm-test-run-nixbot-gitlab> gitlab # [ 2.736123] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. vm-test-run-nixbot-gitlab> gitlab # [ 2.739970] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev vm-test-run-nixbot-gitlab> gitlab # [ 2.743823] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 2.749687] systemd[1]: modprobe@configfs.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.750117] 9pnet: Installing 9P2000 support vm-test-run-nixbot-gitlab> gitlab # [ 2.753959] systemd[1]: Finished Load Kernel Module configfs. vm-test-run-nixbot-gitlab> gitlab # [ 2.760700] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 2.762915] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.767304] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 2.772658] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config vm-test-run-nixbot-gitlab> gitlab # [ 2.778408] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 2.516923] systemd-modules-load[154]: Using 4 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 2.519285] systemd-modules-load[154]: Inserted module 'virtio_balloon' vm-test-run-nixbot-gitlab> gitlab # [ 2.521377] systemd-modules-load[154]: Inserted module 'dm_mod' vm-test-run-nixbot-gitlab> gitlab # [ 2.523341] systemd-modules-load[154]: Inserted module 'virtio_gpu' vm-test-run-nixbot-gitlab> gitlab # [ 2.788863] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 2.791245] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # [ 2.542065] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 2.550561] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 2.556294] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 2.577495] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 2.579971] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 2.581633] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 2.585591] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 2.597527] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 2.606208] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 2.619701] systemd-udevd[193]: Using default interface naming scheme 'v260'. vm-test-run-nixbot-gitlab> gitlab # [ 2.636607] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 2.642211] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 2.643656] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 2.645106] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 3.106270] uhci_hcd 0000:00:01.2: UHCI Host Controller vm-test-run-nixbot-gitlab> gitlab # [ 3.107518] virtio_blk virtio4: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 3.109937] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 vm-test-run-nixbot-gitlab> gitlab # [ 3.113371] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 vm-test-run-nixbot-gitlab> gitlab # [ 3.113901] uhci_hcd 0000:00:01.2: detected 2 ports vm-test-run-nixbot-gitlab> gitlab # [ 3.121402] serio: i8042 KBD port at 0x60,0x64 irq 1 vm-test-run-nixbot-gitlab> gitlab # [ 3.122902] serio: i8042 AUX port at 0x60,0x64 irq 12 vm-test-run-nixbot-gitlab> gitlab # [ 3.124687] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 vm-test-run-nixbot-gitlab> gitlab # [ 3.124789] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 3.125583] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 vm-test-run-nixbot-gitlab> gitlab # [ 3.129796] virtio_blk virtio5: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 3.130972] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 vm-test-run-nixbot-gitlab> gitlab # [ 3.135281] usb usb1: Product: UHCI Host Controller vm-test-run-nixbot-gitlab> gitlab # [ 3.136662] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd vm-test-run-nixbot-gitlab> gitlab # [ 3.138507] usb usb1: SerialNumber: 0000:00:01.2 vm-test-run-nixbot-gitlab> gitlab # [ 3.140256] SCSI subsystem initialized vm-test-run-nixbot-gitlab> gitlab # [ 3.144568] hub 1-0:1.0: USB hub found vm-test-run-nixbot-gitlab> gitlab # [ 3.145645] hub 1-0:1.0: 2 ports detected vm-test-run-nixbot-gitlab> gitlab # [ 3.149759] virtio_blk virtio5: [vdb] 11498104 512-byte logical blocks (5.89 GB/5.48 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 2.932474] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 3.196604] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 vm-test-run-nixbot-gitlab> gitlab # [ 3.210232] scsi host0: ata_piix vm-test-run-nixbot-gitlab> gitlab # [ 2.950673] systemd[1]: Found device /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 2.952531] (udev-worker)[214]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 3.219773] scsi host1: ata_piix vm-test-run-nixbot-gitlab> gitlab # [ 2.956544] (udev-worker)[214]: Network interface NamePolicy=[ 3.220848] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 vm-test-run-nixbot-gitlab> gitlab # disabled on kernel command line.[ 3.223422] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # [ 2.962535] systemd[1]: Found device /dev/disk/by-label/nix-store. vm-test-run-nixbot-gitlab> gitlab # [ 2.966186] (udev-worker)[212]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 2.969819] systemd[1]: Reached target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 2.971485] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 2.975372] systemd-vconsole-setup[227]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 2.978563] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 2.991476] systemd-fsck[242]: nixos: clean, 12/65536 files, 13019/262144 blocks vm-test-run-nixbot-gitlab> gitlab # [ 2.996637] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 3.363749] usb 1-1: new full-speed USB device number 2 using uhci_hcd vm-test-run-nixbot-gitlab> gitlab # [ 3.380232] ata2: found unknown device (class 0) vm-test-run-nixbot-gitlab> gitlab # [ 3.385373] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 vm-test-run-nixbot-gitlab> gitlab # [ 3.392813] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 vm-test-run-nixbot-gitlab> gitlab # [ 3.467964] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray vm-test-run-nixbot-gitlab> gitlab # [ 3.503034] cdrom: Uniform CD-ROM driver Revision: 3.20 vm-test-run-nixbot-gitlab> gitlab # [ 3.531300] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 vm-test-run-nixbot-gitlab> gitlab # [ 3.536634] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 vm-test-run-nixbot-gitlab> gitlab # [ 3.541199] usb 1-1: Product: QEMU USB Tablet vm-test-run-nixbot-gitlab> gitlab # [ 3.544076] usb 1-1: Manufacturer: QEMU vm-test-run-nixbot-gitlab> gitlab # [ 3.546729] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 vm-test-run-nixbot-gitlab> gitlab # [ 3.585947] hid: raw HID events driver (C) Jiri Kosina vm-test-run-nixbot-gitlab> gitlab # [ 3.599675] usbcore: registered new interface driver usbhid vm-test-run-nixbot-gitlab> gitlab # [ 3.603106] usbhid: USB HID core driver vm-test-run-nixbot-gitlab> gitlab # [ 3.616303] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 vm-test-run-nixbot-gitlab> gitlab # [ 3.622242] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 vm-test-run-nixbot-gitlab> gitlab # [ 3.468796] systemd[1]: Mounting /sysroot... vm-test-run-nixbot-gitlab> gitlab # [ 3.907207] EXT4-fs (vda): mounted filesystem b090a739-8458-4c6d-9ecc-d66162d499e6 r/w with ordered data mode. Quota mode: none. vm-test-run-nixbot-gitlab> gitlab # [ 3.648365] systemd[1]: Mounted /sysroot. vm-test-run-nixbot-gitlab> gitlab # [ 3.650335] systemd[1]: Reached target Initrd Root File System. vm-test-run-nixbot-gitlab> gitlab # [ 3.657364] systemd[1]: Mounting /sysroot/nix/.ro-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.663424] systemd[1]: Mounting /sysroot/nix/.rw-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.676104] systemd[1]: Mounting /sysroot/run... vm-test-run-nixbot-gitlab> gitlab # [ 3.684249] systemd[1]: Mounting /sysroot/tmp/shared... vm-test-run-nixbot-gitlab> gitlab # [ 3.952829] erofs (device vdb): mounted with root inode @ nid 128. vm-test-run-nixbot-gitlab> gitlab # [ 3.691125] systemd[1]: Mounting /sysroot/tmp/xchg... vm-test-run-nixbot-gitlab> gitlab # [ 3.957538] 9p: Installing v9fs 9p2000 file system support vm-test-run-nixbot-gitlab> gitlab # [ 3.696994] systemd[1]: Starting Mountpoints Configured in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 3.705481] systemd[1]: Mounted /sysroot/nix/.ro-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.707101] systemd[1]: Mounted /sysroot/nix/.rw-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.708665] systemd[1]: Mounted /sysroot/run. vm-test-run-nixbot-gitlab> gitlab # [ 3.710287] systemd[1]: Mounted /sysroot/tmp/shared. vm-test-run-nixbot-gitlab> gitlab # [ 3.711974] systemd-sysroot-fstab-check[273]: /sysroot should be mounted in the initrd, will request daemon-reload. vm-test-run-nixbot-gitlab> gitlab # [ 3.715590] systemd[1]: Mounted /sysroot/tmp/xchg. vm-test-run-nixbot-gitlab> gitlab # [ 3.716911] systemd[1]: Reload requested from client PID 273 ('systemd-sysroot') (unit initrd-parse-etc.service)... vm-test-run-nixbot-gitlab> gitlab # [ 3.719604] systemd[1]: Reloading... vm-test-run-nixbot-gitlab> gitlab # [ 3.935898] systemd[1]: Reloading finished in 222 ms. vm-test-run-nixbot-gitlab> gitlab # [ 3.984359] systemd-sysroot-fstab-check[273]: Requesting initrd-fs.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 4.114459] systemd-sysroot-fstab-check[273]: Requesting swap.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 4.119682] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 4.126270] systemd[1]: initrd-parse-etc.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.136915] systemd[1]: Finished Mountpoints Configured in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.141253] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. vm-test-run-nixbot-gitlab> gitlab # [ 4.153281] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.161253] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.470413] systemd[1]: Mounting /sysroot/nix/store... vm-test-run-nixbot-gitlab> gitlab # [ 4.531688] systemd[1]: Mounted /sysroot/nix/store. vm-test-run-nixbot-gitlab> gitlab # [ 4.536393] systemd[1]: Reached target Initrd File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.545778] systemd[1]: Starting Find NixOS closure... vm-test-run-nixbot-gitlab> gitlab # [ 4.554677] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 4.589718] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.596825] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.621324] systemd[1]: Finished Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.627154] systemd[1]: Reached target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.634250] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... vm-test-run-nixbot-gitlab> gitlab # [ 4.672455] systemd[1]: Stopped target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.676218] systemd[1]: Stopped target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 4.679506] systemd[1]: Stopped target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 4.682902] systemd[1]: Stopped target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.685908] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.690534] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 4.695180] systemd[1]: Stopped target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.698488] systemd[1]: Stopped target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.701808] systemd[1]: Stopped target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 4.705491] systemd[1]: Stopped target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 4.708504] systemd[1]: Stopped target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.711187] systemd[1]: dbus.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.714183] systemd[1]: Closed D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.717161] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.720824] systemd[1]: Stopped Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.723294] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio vm-test-run-nixbot-gitlab> gitlab # [ 4.727944] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 4.729759] systemd[1]: systemd-sysctl.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.731805] systemd[1]: Stopped Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 4.733349] systemd[1]: systemd-modules-load.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.735167] systemd[1]: Stopped Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 4.736462] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.738401] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.743457] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.745290] systemd[1]: Stopped Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 4.749666] systemd[1]: Stopped target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.751057] systemd[1]: Stopped target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.752695] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.754463] systemd[1]: Stopped Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 4.759250] systemd[1]: Stopping Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 4.763219] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.766467] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 4.772491] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.776316] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.782747] systemd[1]: systemd-udevd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.786697] systemd[1]: Stopped Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 4.788767] systemd[1]: systemd-udevd.service: Consumed 1.146s CPU time over 2.185s wall clock time, 24.4M memory peak. vm-test-run-nixbot-gitlab> gitlab # [ 4.798583] systemd[1]: initrd-cleanup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.804114] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. vm-test-run-nixbot-gitlab> gitlab # [ 4.813961] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.816135] systemd[1]: Closed udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.822714] systemd[1]: Starting Cleanup udev Database... vm-test-run-nixbot-gitlab> gitlab # [ 4.824445] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.826548] systemd[1]: Stopped Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 4.829524] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.831725] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 4.837286] systemd[1]: kmod-static-nodes.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.839480] systemd[1]: Stopped Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 4.857481] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.862323] systemd[1]: Finished Cleanup udev Database. vm-test-run-nixbot-gitlab> gitlab # [ 4.867117] systemd[1]: Reached target Switch Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.869704] systemd[1]: Starting NixOS Activation... vm-test-run-nixbot-gitlab> gitlab # [ 4.990728] initrd-nixos-activation-start[563]: booting system configuration /nix/store/vg6nqmxghq3ys5636j4rn33azchwqhfb-nixos-system-gitlab-test vm-test-run-nixbot-gitlab> gitlab # [ 5.034642] initrd-nixos-activation-start[563]: running activation script... vm-test-run-nixbot-gitlab> gitlab # [ 5.266675] initrd-nixos-activation-start[586]: setting up /etc... vm-test-run-nixbot-gitlab> gitlab # [ 5.374861] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 5.379634] systemd[1]: Finished NixOS Activation. vm-test-run-nixbot-gitlab> gitlab # [ 5.386476] systemd[1]: Starting Switch Root... vm-test-run-nixbot-gitlab> gitlab # [ 5.421154] systemd[1]: Switching root. vm-test-run-nixbot-gitlab> gitlab # [ 6.076719] systemd-journald[151]: Received SIGTERM from PID 1 (systemd). vm-test-run-nixbot-gitlab> gitlab # [ 6.228712] NET: Registered PF_VSOCK protocol family vm-test-run-nixbot-gitlab> gitlab # [ 6.598029] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 6.614106] systemd[1]: Detected virtualization kvm. vm-test-run-nixbot-gitlab> gitlab # [ 6.617631] systemd[1]: Detected architecture x86-64. vm-test-run-nixbot-gitlab> gitlab # [ 6.621339] systemd[1]: Detected first boot. vm-test-run-nixbot-gitlab> gitlab # [ 6.625951] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 6.767512] systemd[1]: bpf-restrict-fs: LSM BPF program attached vm-test-run-nixbot-gitlab> gitlab # [ 6.840158] systemd[1]: Applying preset policy. vm-test-run-nixbot-gitlab> gitlab # [ 7.009107] systemd[1]: Populated /etc with preset unit settings. vm-test-run-nixbot-gitlab> gitlab # [ 7.127630] systemd[1]: initrd-switch-root.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.129973] systemd[1]: Stopped initrd-switch-root.service. vm-test-run-nixbot-gitlab> gitlab # [ 7.133294] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. vm-test-run-nixbot-gitlab> gitlab # [ 7.136831] systemd[1]: Created slice Slice /system/getty. vm-test-run-nixbot-gitlab> gitlab # [ 7.138979] systemd[1]: Created slice GitLab DevOps Platform Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.141280] systemd[1]: Created slice User and Session Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.142921] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.144956] systemd[1]: Started Forward Password Requests to Wall Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.146873] systemd[1]: Expecting device /dev/hvc0... vm-test-run-nixbot-gitlab> gitlab # [ 7.148179] systemd[1]: Expecting device /dev/ttyS0... vm-test-run-nixbot-gitlab> gitlab # [ 7.149581] systemd[1]: Reached target Local Encrypted Volumes. vm-test-run-nixbot-gitlab> gitlab # [ 7.151036] systemd[1]: Stopped target initrd-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.152399] systemd[1]: Stopped target initrd-root-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.153875] systemd[1]: Stopped target initrd-switch-root.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.155426] systemd[1]: Reached target Virtual Machines and Containers. vm-test-run-nixbot-gitlab> gitlab # [ 7.157172] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.158514] systemd[1]: Reached target Remote File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.159995] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.161317] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 7.163651] systemd[1]: Listening on Process Core Dump Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.165869] systemd[1]: Listening on Credential Encryption/Decryption. vm-test-run-nixbot-gitlab> gitlab # [ 7.169286] systemd[1]: Starting Journal Log Access Socket... vm-test-run-nixbot-gitlab> gitlab # [ 7.171631] systemd[1]: Listening on Journal Audit Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.173348] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.175280] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.177801] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.181493] systemd[1]: Mounting Huge Pages File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.184674] systemd[1]: Mounting POSIX Message Queue File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.187924] systemd[1]: Mounting Kernel Debug File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.190735] systemd[1]: Mounting Kernel Trace File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.198553] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 7.200390] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio vm-test-run-nixbot-gitlab> gitlab # [ 7.204790] systemd[1]: Starting Load Kernel Module configfs... vm-test-run-nixbot-gitlab> gitlab # [ 7.206544] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm vm-test-run-nixbot-gitlab> gitlab # [ 7.209679] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.212669] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse vm-test-run-nixbot-gitlab> gitlab # [ 7.218811] systemd[1]: Mounting FUSE Control File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.220372] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 7.231572] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 7.234492] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 7.260690] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... vm-test-run-nixbot-gitlab> gitlab # [ 7.264348] systemd[1]: Starting Remount Root and Kernel File Systems... vm-test-run-nixbot-gitlab> gitlab # [ 7.266119] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.272824] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 7.276067] systemd-journald[782]: Collecting audit messages is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 7.287054] systemd[1]: Listening on Journal Log Access Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.288779] systemd[1]: Mounted Huge Pages File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.288844] loop: module loaded vm-test-run-nixbot-gitlab> gitlab # [ 7.289353] systemd[1]: Mounted POSIX Message Queue File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.292930] systemd[1]: Mounted Kernel Debug File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.294582] systemd[1]: Mounted Kernel Trace File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.294663] EXT4-fs (vda): re-mounted b090a739-8458-4c6d-9ecc-d66162d499e6. vm-test-run-nixbot-gitlab> gitlab # [ 7.036116] systemd[1]: Queued start job for default target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 7.299248] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 7.038535] systemd[1]: Unnecessary job was removed for /dev/vdb. vm-test-run-nixbot-gitlab> gitlab # [ 7.041364] systemd[1]: systemd-journald.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.043651] systemd-modules-load[783]: Using 3 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 7.045469] systemd-modules-load[783]: Inserted module 'tls' vm-test-run-nixbot-gitlab> gitlab # [ 7.308709] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # [ 7.047479] systemd-modules-load[783]: Inserted module 'loop' vm-test-run-nixbot-gitlab> gitlab # [ 7.050829] systemd[1]: modprobe@configfs.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.052912] systemd[1]: Finished Load Kernel Module configfs. vm-test-run-nixbot-gitlab> gitlab # [ 7.055051] systemd[1]: Mounted FUSE Control File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.057299] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 7.058834] systemd-oomd[785]: No swap; memory pressure usage will be degraded vm-test-run-nixbot-gitlab> gitlab # [ 7.061275] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. vm-test-run-nixbot-gitlab> gitlab # [ 7.063519] systemd[1]: Finished Remount Root and Kernel File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.075821] systemd[1]: Mounting Kernel Configuration File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.078671] systemd[1]: Starting Firewall... vm-test-run-nixbot-gitlab> gitlab # [ 7.088239] systemd[1]: Starting Flush Journal to Persistent Storage... vm-test-run-nixbot-gitlab> gitlab # [ 7.092241] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.095532] systemd[1]: Starting Load/Save OS Random Seed... vm-test-run-nixbot-gitlab> gitlab # [ 7.098320] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 7.105215] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 7.107919] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.110393] systemd[1]: Mounted Kernel Configuration File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.377060] systemd-journald[782]: Received client request to flush runtime journal. vm-test-run-nixbot-gitlab> gitlab # [ 7.179964] systemd[1]: Finished Load/Save OS Random Seed. vm-test-run-nixbot-gitlab> gitlab # [ 7.182890] systemd[1]: Reached target First Boot Complete. vm-test-run-nixbot-gitlab> gitlab # [ 7.185051] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 7.186764] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 7.188655] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 7.190469] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 7.192219] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.194206] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 7.196249] systemd[1]: Finished Flush Journal to Persistent Storage. vm-test-run-nixbot-gitlab> gitlab # [ 7.211329] systemd-udevd[825]: Using default interface naming scheme 'v260'. vm-test-run-nixbot-gitlab> gitlab # [ 7.237275] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 7.252146] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 7.372128] systemd[1]: Finished Firewall. vm-test-run-nixbot-gitlab> gitlab # [ 7.378222] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse vm-test-run-nixbot-gitlab> gitlab # [ 7.416431] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.444472] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.485811] (udev-worker)[915]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 7.492853] (udev-worker)[917]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 7.496550] (udev-worker)[917]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 7.773317] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console vm-test-run-nixbot-gitlab> gitlab # [ 7.773356] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 vm-test-run-nixbot-gitlab> gitlab # [ 7.791631] Console: switching to colour dummy device 80x25 vm-test-run-nixbot-gitlab> gitlab # [ 7.793569] [drm] Found bochs VGA, ID 0xb0c5. vm-test-run-nixbot-gitlab> gitlab # [ 7.794397] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. vm-test-run-nixbot-gitlab> gitlab # [ 7.800243] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic vm-test-run-nixbot-gitlab> gitlab # [ 7.801596] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 vm-test-run-nixbot-gitlab> gitlab # [ 7.803810] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 vm-test-run-nixbot-gitlab> gitlab # [ 7.813954] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 vm-test-run-nixbot-gitlab> gitlab # [ 7.815739] fbcon: bochs-drmdrmfb (fb0) is primary device vm-test-run-nixbot-gitlab> gitlab # [ 7.820537] ACPI: button: Power Button [PWRF] vm-test-run-nixbot-gitlab> gitlab # [ 7.825869] Console: switching to colour frame buffer device 160x50 vm-test-run-nixbot-gitlab> gitlab # [ 7.829989] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device vm-test-run-nixbot-gitlab> gitlab # [ 7.830673] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 vm-test-run-nixbot-gitlab> gitlab # [ 7.834722] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD vm-test-run-nixbot-gitlab> gitlab # [ 7.844712] rtc_cmos 00:05: RTC can wake from S4 vm-test-run-nixbot-gitlab> gitlab # [ 7.582469] systemd[1]: Condition check resulted in Virtio network device being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.584942] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.851384] rtc_cmos 00:05: registered as rtc0 vm-test-run-nixbot-gitlab> gitlab # [ 7.588112] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 7.856054] rtc_cmos 00:05: setting system clock to 2026-06-29T22:05:04 UTC (1782770704) vm-test-run-nixbot-gitlab> gitlab # [ 7.856553] systemd-journald[782]: Time jumped backwards, rotating. vm-test-run-nixbot-gitlab> gitlab # [ 7.858314] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs vm-test-run-nixbot-gitlab> gitlab # [ 7.862944] parport_pc 00:03: reported by Plug and Play ACPI vm-test-run-nixbot-gitlab> gitlab # [ 7.864098] Floppy drive(s): fd0 is 2.88M AMI BIOS vm-test-run-nixbot-gitlab> gitlab # [ 7.608315] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.878579] FDC 0 is a S82078B vm-test-run-nixbot-gitlab> gitlab # [ 7.620382] systemd[1]: Early TPM SRK Setup skipped, unmet co[ 7.885015] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] vm-test-run-nixbot-gitlab> gitlab # ndition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.625335] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.903914] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 vm-test-run-nixbot-gitlab> gitlab # [ 8.036003] mousedev: PS/2 mouse device common for all mice vm-test-run-nixbot-gitlab> gitlab # [ 8.040508] ppdev: user-space parallel port driver vm-test-run-nixbot-gitlab> gitlab # [ 7.792304] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 7.803511] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.805609] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 7.807616] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 8.080908] kvm_amd: TSC scaling supported vm-test-run-nixbot-gitlab> gitlab # [ 8.082023] kvm_amd: Nested Virtualization enabled vm-test-run-nixbot-gitlab> gitlab # [ 8.083540] kvm_amd: Nested Paging enabled vm-test-run-nixbot-gitlab> gitlab # [ 8.084334] kvm_amd: LBR virtualization supported vm-test-run-nixbot-gitlab> gitlab # [ 8.085666] kvm_amd: Virtual VMLOAD VMSAVE supported vm-test-run-nixbot-gitlab> gitlab # [ 8.086927] kvm_amd: Virtual GIF supported vm-test-run-nixbot-gitlab> gitlab # [ 8.088025] kvm_amd: Virtual NMI enabled vm-test-run-nixbot-gitlab> gitlab # [ 8.092660] erofs (device vdb): failed to read inode meta block (nid: 109346528): -4 vm-test-run-nixbot-gitlab> gitlab # [ 7.836501] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.838521] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 7.840975] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 7.868435] systemd[1]: Mounting /run/wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 8.145008] EDAC MC: Ver: 3.0.0 vm-test-run-nixbot-gitlab> gitlab # [ 7.887495] systemd[1]: Mounted /run/wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 7.889413] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.891129] systemd[1]: Listening on Boot Loader Control Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.893388] systemd[1]: Starting register-nix-paths.service... vm-test-run-nixbot-gitlab> gitlab # [ 7.895134] systemd[1]: Starting Create SUID/SGID Wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 7.897045] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. vm-test-run-nixbot-gitlab> gitlab # [ 7.899829] systemd[1]: Starting Save Transient machine-id to Disk... vm-test-run-nixbot-gitlab> gitlab # [ 7.902218] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 7.925656] systemd-vconsole-setup[956]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 7.931332] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 7.937813] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 7.942952] systemd[1]: Starting Rebuild Journal Catalog... vm-test-run-nixbot-gitlab> gitlab # [ 7.945252] systemd[1]: Starting Record System Boot/Shutdown in UTMP... vm-test-run-nixbot-gitlab> gitlab # [ 7.952107] systemd[1]: Finished Save Transient machine-id to Disk. vm-test-run-nixbot-gitlab> gitlab # [ 7.964414] systemd[1]: Finished Record System Boot/Shutdown in UTMP. vm-test-run-nixbot-gitlab> gitlab # [ 7.978670] systemd[1]: Finished Rebuild Journal Catalog. vm-test-run-nixbot-gitlab> gitlab # [ 7.980881] systemd[1]: Starting Update is Completed... vm-test-run-nixbot-gitlab> gitlab # [ 7.996098] systemd[1]: Finished Update is Completed. vm-test-run-nixbot-gitlab> gitlab # [ 8.061921] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.063847] systemd[1]: Finished Create SUID/SGID Wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 8.065279] systemd[1]: suid-sgid-wrappers.service: Consumed 123ms CPU time over 167ms wall clock time, 4.6M memory peak, 1.1M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.306143] systemd[1]: Finished register-nix-paths.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.308299] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 8.310393] systemd[1]: Started Discard unused filesystem blocks once a week. vm-test-run-nixbot-gitlab> gitlab # [ 8.312681] systemd[1]: Started logrotate.timer. vm-test-run-nixbot-gitlab> gitlab # [ 8.314422] systemd[1]: Started Daily Cleanup of Temporary Directories. vm-test-run-nixbot-gitlab> gitlab # [ 8.317431] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 8.319372] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.321615] systemd[1]: Listening on Nix Daemon Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.329422] systemd[1]: Starting nixbot CI service socket... vm-test-run-nixbot-gitlab> gitlab # [ 8.331734] systemd[1]: Listening on Hostname Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.333729] systemd[1]: Listening on nixbot CI service socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.335701] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 8.338147] systemd[1]: Starting D-Bus System Message Bus... vm-test-run-nixbot-gitlab> gitlab # [ 8.402718] dbus-broker-launch[1046]: Looking up NSS user entry for 'systemd-timesync'... vm-test-run-nixbot-gitlab> gitlab # [ 8.408829] dbus-broker-launch[1046]: NSS returned no entry for 'systemd-timesync' vm-test-run-nixbot-gitlab> gitlab # [ 8.414112] dbus-broker-launch[1046]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" vm-test-run-nixbot-gitlab> gitlab # [ 8.424142] systemd[1]: Started D-Bus System Message Bus. vm-test-run-nixbot-gitlab> gitlab # [ 8.427624] dbus-broker-launch[1046]: Ready vm-test-run-nixbot-gitlab> gitlab # [ 8.435742] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 8.441249] systemd[1]: Started backdoor.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.458910] systemd[1]: Starting gitlab-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.462964] systemd[1]: Starting Import lastlog data into lastlog2 database... vm-test-run-nixbot-gitlab> gitlab # [ 8.469107] systemd[1]: Starting Logrotate configuration check... vm-test-run-nixbot-gitlab> gitlab # [ 8.473666] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.477791] systemd[1]: Starting Post-Boot Actions... vm-test-run-nixbot-gitlab> gitlab # connecting to host... vm-test-run-nixbot-gitlab> gitlab # [ 8.485738] systemd[1]: Started Reset console on configuration changes. vm-test-run-nixbot-gitlab> gitlab # [ 8.487472] systemd[1]: Starting resolvconf update... vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n' vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.36 seconds) vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.36 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 8.505510] logrotate[1051]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. vm-test-run-nixbot-gitlab> gitlab # [ 8.506677] logrotate[1051]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf vm-test-run-nixbot-gitlab> gitlab # [ 8.510591] logrotate[1051]: note: 'monthly' overrides previously specified 'weekly' vm-test-run-nixbot-gitlab> gitlab # [ 8.511925] logrotate[1051]: Reading state from file: /var/lib/logrotate.status vm-test-run-nixbot-gitlab> gitlab # [ 8.512582] logrotate[1051]: state file /var/lib/logrotate.status does not exist vm-test-run-nixbot-gitlab> gitlab # [ 8.513684] logrotate[1051]: Allocating hash table for state file, size 64 entries vm-test-run-nixbot-gitlab> gitlab # [ 8.514889] logrotate[1051]: Handling 4 logs vm-test-run-nixbot-gitlab> gitlab # [ 8.515693] logrotate[1051]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.517163] logrotate[1051]: considering log /var/log/btmp vm-test-run-nixbot-gitlab> gitlab # [ 8.517976] logrotate[1051]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.520248] logrotate[1051]: Now: 2026-06-29 22:05 vm-test-run-nixbot-gitlab> gitlab # [ 8.521166] logrotate[1051]: Last rotated at 2026-06-29 22:00 vm-test-run-nixbot-gitlab> gitlab # [ 8.522108] logrotate[1051]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 8.522985] logrotate[1051]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.525918] logrotate[1051]: considering log /var/log/wtmp vm-test-run-nixbot-gitlab> gitlab # [ 8.527116] logrotate[1051]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.528090] logrotate[1051]: Now: 2026-06-29 22:05 vm-test-run-nixbot-gitlab> gitlab # [ 8.528987] logrotate[1051]: Last rotated at 2026-06-29 22:00 vm-test-run-nixbot-gitlab> gitlab # [ 8.530275] logrotate[1051]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 8.531275] logrotate[1051]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.532175] logrotate[1051]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1051) vm-test-run-nixbot-gitlab> gitlab # [ 8.533140] logrotate[1051]: considering log /var/gitlab/state/log/*.log vm-test-run-nixbot-gitlab> gitlab # [ 8.534109] logrotate[1051]: log /var/gitlab/state/log/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 8.535058] logrotate[1051]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.535974] logrotate[1051]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1051) vm-test-run-nixbot-gitlab> gitlab # [ 8.537280] logrotate[1051]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.538215] logrotate[1051]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1051) vm-test-run-nixbot-gitlab> gitlab # [ 8.539166] logrotate[1051]: considering log /var/log/nginx/*.log vm-test-run-nixbot-gitlab> gitlab # [ 8.540138] logrotate[1051]: log /var/log/nginx/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 8.541090] logrotate[1051]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.541969] logrotate[1051]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1051) vm-test-run-nixbot-gitlab> gitlab # [ 8.566704] systemd[1]: Finished Logrotate configuration check. vm-test-run-nixbot-gitlab> gitlab # [ 8.570244] systemd[1]: Finished Post-Boot Actions. vm-test-run-nixbot-gitlab> gitlab # [ 8.578207] systemd[1]: Finished Import lastlog data into lastlog2 database. vm-test-run-nixbot-gitlab> gitlab # [ 8.591764] nsncd[1054]: Jun 29 22:05:05.497 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 8.592554] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.595707] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.599630] systemd[1]: nscd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.601133] systemd[1]: Stopped Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.601856] systemd[1]: nscd.service: Consumed 25ms CPU time over 126ms wall clock time, 8M memory peak, 5.7M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.609889] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.628078] systemd[1]: Finished resolvconf update. vm-test-run-nixbot-gitlab> gitlab # [ 8.629800] systemd[1]: Reached target Preparation for Network. vm-test-run-nixbot-gitlab> gitlab # [ 8.632087] nsncd[1129]: Jun 29 22:05:05.538 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 8.634403] systemd[1]: Starting DHCP Client... vm-test-run-nixbot-gitlab> gitlab # [ 8.636168] systemd[1]: Starting Address configuration of eth1... vm-test-run-nixbot-gitlab> gitlab # [ 8.639431] systemd[1]: Starting Extra networking commands.... vm-test-run-nixbot-gitlab> gitlab # [ 8.640951] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.642683] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.644311] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.651330] systemd[1]: Starting User Login Management... vm-test-run-nixbot-gitlab> gitlab # [ 8.669959] network-addresses-eth1-start[1154]: adding address 192.168.1.1/24... done vm-test-run-nixbot-gitlab> gitlab # [ 8.676823] network-addresses-eth1-start[1154]: adding address 2001:db8:1::1/64... done vm-test-run-nixbot-gitlab> gitlab # [ 8.685908] systemd[1]: Finished Address configuration of eth1. vm-test-run-nixbot-gitlab> gitlab # [ 8.702347] systemd-logind[1156]: New seat seat0. vm-test-run-nixbot-gitlab> gitlab # [ 8.707511] systemd-logind[1156]: Watching system buttons on /dev/input/event2 (Power Button) vm-test-run-nixbot-gitlab> gitlab # [ 8.709159] systemd-logind[1156]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 8.710922] systemd-logind[1156]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 8.712573] systemd[1]: Started User Login Management. vm-test-run-nixbot-gitlab> gitlab # [ 8.717985] systemd[1]: Starting linger-users.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.729605] systemd[1]: Finished Extra networking commands.. vm-test-run-nixbot-gitlab> gitlab # [ 8.732184] systemd[1]: Reached target Network. vm-test-run-nixbot-gitlab> gitlab # [ 8.739311] systemd[1]: Starting gitlab-workhorse.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.744325] systemd[1]: Starting Nginx Web Server... vm-test-run-nixbot-gitlab> gitlab # [ 8.747478] systemd[1]: Starting PostgreSQL Server... vm-test-run-nixbot-gitlab> gitlab # [ 8.753659] dhcpcd[1181]: dhcpcd-10.3.2 starting vm-test-run-nixbot-gitlab> gitlab # [ 8.756139] systemd[1]: Starting Redis Server - redis-gitlab... vm-test-run-nixbot-gitlab> gitlab # [ 8.757792] systemd[1]: Starting Permit User Sessions... vm-test-run-nixbot-gitlab> gitlab # [ 8.759313] dhcpcd[1226]: dev: loaded udev vm-test-run-nixbot-gitlab> gitlab # [ 8.760041] systemd[1]: linger-users.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.760576] systemd[1]: Finished linger-users.service. vm-test-run-nixbot-gitlab> gitlab # [ 9.027851] 8021q: 802.1Q VLAN Support v1.8 vm-test-run-nixbot-gitlab> gitlab # [ 9.029624] 8021q: adding VLAN 0 to HW filter on device eth1 vm-test-run-nixbot-gitlab> gitlab # [ 8.784086] systemd[1]: Finished Permit User Sessions. vm-test-run-nixbot-gitlab> gitlab # [ 8.790130] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. vm-test-run-nixbot-gitlab> gitlab # [ 8.794679] systemd[1]: Started Getty on tty1. vm-test-run-nixbot-gitlab> gitlab # [ 8.796853] systemd[1]: Reached target Login Prompts. vm-test-run-nixbot-gitlab> gitlab # [ 8.866844] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 9.141143] cfg80211: Loading compiled-in X.509 certificates for regulatory database vm-test-run-nixbot-gitlab> gitlab # [ 9.148537] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' vm-test-run-nixbot-gitlab> gitlab # [ 9.150306] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' vm-test-run-nixbot-gitlab> gitlab # [ 9.153067] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 vm-test-run-nixbot-gitlab> gitlab # [ 8.893185] nginx-pre-start[1244]: nginx: the[ 9.157324] cfg80211: failed to load regulatory.db vm-test-run-nixbot-gitlab> gitlab # configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok vm-test-run-nixbot-gitlab> gitlab # [ 8.898261] nginx-pre-start[1244]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful vm-test-run-nixbot-gitlab> gitlab # [ 8.901738] systemd[1]: Started Nginx Web Server. vm-test-run-nixbot-gitlab> gitlab # [ 8.925841] redis-server[1246]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. vm-test-run-nixbot-gitlab> gitlab # [ 8.929336] redis-server[1246]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo vm-test-run-nixbot-gitlab> gitlab # [ 8.929944] redis-server[1246]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1246, just started vm-test-run-nixbot-gitlab> gitlab # [ 8.932509] redis-server[1246]: Configuration loaded vm-test-run-nixbot-gitlab> gitlab # [ 8.934177] redis[1246]: monotonic clock: POSIX clock_gettime vm-test-run-nixbot-gitlab> gitlab # [ 8.935366] systemd[1]: Started Redis Server - redis-gitlab. vm-test-run-nixbot-gitlab> gitlab # [ 8.935964] redis[1246]: Running mode=standalone, port=0. vm-test-run-nixbot-gitlab> gitlab # [ 8.937161] redis[1246]: Server initialized vm-test-run-nixbot-gitlab> gitlab # [ 8.938234] redis[1246]: Ready to accept connections unix vm-test-run-nixbot-gitlab> gitlab # [ 8.939140] redis[1246]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. vm-test-run-nixbot-gitlab> gitlab # [ 9.209742] 8021q: adding VLAN 0 to HW filter on device eth0 vm-test-run-nixbot-gitlab> gitlab # [ 8.948939] postgresql-pre-start[1249]: The files belonging to this database system will be owned by user "postgres". vm-test-run-nixbot-gitlab> gitlab # [ 8.950217] postgresql-pre-start[1249]: This user must also own the server process. vm-test-run-nixbot-gitlab> gitlab # [ 8.952535] dhcpcd[1226]: eth0: waiting for carrier vm-test-run-nixbot-gitlab> gitlab # [ 8.953928] dhcpcd[1226]: eth0: carrier acquired vm-test-run-nixbot-gitlab> gitlab # [ 8.956270] dhcpcd[1226]: DUID 00:01:00:01:31:d5:a8:91:52:54:00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 8.956812] dhcpcd[1226]: eth0: IAID 00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 8.958083] postgresql-pre-start[1249]: The database cluster will be initialized with locale "en_US.UTF-8". vm-test-run-nixbot-gitlab> gitlab # [ 8.958878] postgresql-pre-start[1249]: The default database encoding has accordingly been set to "UTF8". vm-test-run-nixbot-gitlab> gitlab # [ 8.960112] postgresql-pre-start[1249]: The default text search configuration will be set to "english". vm-test-run-nixbot-gitlab> gitlab # [ 8.960958] postgresql-pre-start[1249]: Data page checksums are disabled. vm-test-run-nixbot-gitlab> gitlab # [ 8.961950] postgresql-pre-start[1249]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok vm-test-run-nixbot-gitlab> gitlab # [ 8.963056] postgresql-pre-start[1249]: creating subdirectories ... ok vm-test-run-nixbot-gitlab> gitlab # [ 8.963944] postgresql-pre-start[1249]: selecting dynamic shared memory implementation ... posix vm-test-run-nixbot-gitlab> gitlab # [ 8.965230] dhcpcd[1226]: eth0: adding address fe80::5054:ff:fe12:3456 vm-test-run-nixbot-gitlab> gitlab # [ 8.985552] postgresql-pre-start[1249]: selecting default "max_connections" ... 100 vm-test-run-nixbot-gitlab> gitlab # [ 9.019405] postgresql-pre-start[1249]: selecting default "shared_buffers" ... 128MB vm-test-run-nixbot-gitlab> gitlab # [ 9.276965] systemd[1]: Started gitlab-workhorse.service. vm-test-run-nixbot-gitlab> gitlab # [ 9.396764] postgresql-pre-start[1249]: selecting default time zone ... UTC vm-test-run-nixbot-gitlab> gitlab # [ 9.398572] postgresql-pre-start[1249]: creating configuration files ... ok vm-test-run-nixbot-gitlab> gitlab # [ 9.544475] gitlab-workhorse[1282]: time="2026-06-29T22:05:06Z" level=info msg=Starting build_time=19700101.000000 version=18.11.4 vm-test-run-nixbot-gitlab> gitlab # [ 9.544797] gitlab-workhorse[1282]: time="2026-06-29T22:05:06Z" level=info msg="Using redis/go-redis" vm-test-run-nixbot-gitlab> gitlab # [ 9.547699] gitlab-workhorse[1282]: time="2026-06-29T22:05:06Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix vm-test-run-nixbot-gitlab> gitlab # [ 9.582528] postgresql-pre-start[1249]: running bootstrap script ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.054441] postgresql-pre-start[1249]: performing post-bootstrap initialization ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.390812] postgresql-pre-start[1249]: syncing data to disk ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.394935] postgresql-pre-start[1249]: initdb: warning: enabling "trust" authentication for local connections vm-test-run-nixbot-gitlab> gitlab # [ 10.401270] postgresql-pre-start[1249]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. vm-test-run-nixbot-gitlab> gitlab # [ 10.411333] postgresql-pre-start[1249]: Success. You can now start the database server using: vm-test-run-nixbot-gitlab> gitlab # [ 10.416360] postgresql-pre-start[1249]: pg_ctl -D /var/lib/postgresql/17 -l logfile start vm-test-run-nixbot-gitlab> gitlab # [ 10.518689] postgres[1306]: [1306] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit vm-test-run-nixbot-gitlab> gitlab # [ 10.526619] postgres[1306]: [1306] LOG: listening on IPv6 address "::1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 10.531850] postgres[1306]: [1306] LOG: listening on IPv4 address "127.0.0.1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 10.538227] postgres[1306]: [1306] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" vm-test-run-nixbot-gitlab> gitlab # [ 10.544885] postgres[1313]: [1313] LOG: database system was shut down at 2026-06-29 22:05:06 GMT vm-test-run-nixbot-gitlab> gitlab # [ 10.568640] postgres[1306]: [1306] LOG: database system is ready to accept connections vm-test-run-nixbot-gitlab> gitlab # [ 10.578378] systemd[1]: Started PostgreSQL Server. vm-test-run-nixbot-gitlab> gitlab # [ 10.581632] systemd[1]: Starting PostgreSQL Setup Scripts... vm-test-run-nixbot-gitlab> gitlab # [ 10.774119] dhcpcd[1226]: eth0: soliciting a DHCP lease vm-test-run-nixbot-gitlab> gitlab # [ 10.781292] postgresql-setup-start[1328]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 11.048804] NET: Registered PF_PACKET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 10.793682] dhcpcd[1226]: eth0: offered 10.0.2.15 from 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 10.798874] dhcpcd[1226]: eth0: probing address 10.0.2.15/24 vm-test-run-nixbot-gitlab> gitlab # [ 10.822481] postgresql-setup-start[1339]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 10.836288] postgresql-setup-start[1341]: ALTER DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 10.858264] postgresql-setup-start[1347]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 10.861661] systemd[1]: Finished PostgreSQL Setup Scripts. vm-test-run-nixbot-gitlab> gitlab # [ 10.863303] systemd[1]: Reached target PostgreSQL. vm-test-run-nixbot-gitlab> gitlab # [ 10.864784] systemd[1]: Starting gitlab-postgresql.service... vm-test-run-nixbot-gitlab> gitlab # [ 10.874954] dhcpcd[1226]: eth0: soliciting an IPv6 router vm-test-run-nixbot-gitlab> gitlab # [ 10.876749] dhcpcd[1226]: eth0: Router Advertisement from fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 10.878392] dhcpcd[1226]: eth0: adding address fec0::5054:ff:fe12:3456/64 vm-test-run-nixbot-gitlab> gitlab # [ 10.880164] dhcpcd[1226]: eth0: adding route to fec0::/64 vm-test-run-nixbot-gitlab> gitlab # [ 10.881800] dhcpcd[1226]: eth0: adding default route via fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 10.972894] gitlab-postgresql-start[1356]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 11.013993] gitlab-postgresql-start[1361]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 11.070521] gitlab-postgresql-start[1366]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 11.072358] systemd[1]: Finished gitlab-postgresql.service. vm-test-run-nixbot-gitlab> gitlab # [ 11.174877] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1365]: mkdir -p /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 11.183203] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1365]: chmod 700 /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 11.276369] systemd[1]: Finished gitlab-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 11.282290] systemd[1]: Started gitaly.service. vm-test-run-nixbot-gitlab> gitlab # [ 11.287738] systemd[1]: Starting gitlab-db-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 11.820190] gitaly[1382]: time="2026-06-29T22:05:08.727Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 11.820571] gitaly[1382]: time="2026-06-29T22:05:08.727Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 11.823242] gitaly[1382]: time="2026-06-29T22:05:08.727Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 11.825812] gitaly[1382]: time="2026-06-29T22:05:08.727Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 11.828591] gitaly[1382]: time="2026-06-29T22:05:08.727Z" level=info msg="Starting Gitaly" pid=1382 version=18.11.4 vm-test-run-nixbot-gitlab> gitlab # [ 12.038158] gitaly[1382]: time="2026-06-29T22:05:08.945Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=217 pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.038972] gitaly[1382]: time="2026-06-29T22:05:08.945Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.040225] gitaly[1382]: time="2026-06-29T22:05:08.946Z" level=info msg="finished initializing command factory" duration_ms=0 pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.041278] gitaly[1382]: time="2026-06-29T22:05:08.946Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1382/git-exec-551193852.d/git pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.049301] gitaly[1382]: time="2026-06-29T22:05:08.956Z" level=info msg="finished detecting git version" duration_ms=10 pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.050113] gitaly[1382]: time="2026-06-29T22:05:08.956Z" level=info msg="using Git version" pid=1382 version=2.53.g vm-test-run-nixbot-gitlab> gitlab # [ 12.052149] gitaly[1382]: time="2026-06-29T22:05:08.957Z" level=info msg="clearing disk cache object folder" pid=1382 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 12.053119] gitaly[1382]: time="2026-06-29T22:05:08.957Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache182123026 pid=1382 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 12.054053] gitaly[1382]: time="2026-06-29T22:05:08.957Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache182123026 pid=1382 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 12.054799] gitaly[1382]: time="2026-06-29T22:05:08.957Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.055425] gitaly[1382]: time="2026-06-29T22:05:08.957Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.056878] gitaly[1382]: time="2026-06-29T22:05:08.957Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache182123026 clear_duration_ms=0 pid=1382 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 12.138679] gitaly[1382]: time="2026-06-29T22:05:09.045Z" level=info msg="counting repositories" pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.139262] gitaly[1382]: time="2026-06-29T22:05:09.045Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.140111] gitaly[1382]: time="2026-06-29T22:05:09.045Z" level=info msg="finished tempdir cleaner walk" pid=1382 storage=default time_ms=0 vm-test-run-nixbot-gitlab> gitlab # [ 12.140961] gitaly[1382]: time="2026-06-29T22:05:09.045Z" level=info msg="starting to count repositories" pid=1382 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 12.142377] gitaly[1382]: time="2026-06-29T22:05:09.045Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1382 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 12.143042] gitaly[1382]: time="2026-06-29T22:05:09.045Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1382 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 12.143887] gitaly[1382]: time="2026-06-29T22:05:09.045Z" level=info msg="completed counting all repositories" counting_duration=0 pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.145109] gitaly[1382]: time="2026-06-29T22:05:09.045Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1382/sock.d/intern pid=1382 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 12.146042] gitaly[1382]: time="2026-06-29T22:05:09.048Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.146857] gitaly[1382]: time="2026-06-29T22:05:09.049Z" level=info msg="Started Gitaly" duration_ms=321 pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 12.147967] gitaly[1382]: time="2026-06-29T22:05:09.049Z" level=info msg="maintenance: daily scheduled" pid=1382 scheduled="2026-06-30 12:00:00 +0000 UTC" vm-test-run-nixbot-gitlab> gitlab # [ 12.767453] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1385]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 12.795665] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1385]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 12.810871] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1385]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 13.991191] gitaly[1382]: time="2026-06-29T22:05:10.897Z" level=info msg="License database preloaded" duration_ms=2169 pid=1382 vm-test-run-nixbot-gitlab> gitlab # [ 16.384170] dhcpcd[1226]: eth0: leased 10.0.2.15 for 86400 seconds vm-test-run-nixbot-gitlab> gitlab # [ 16.384808] dhcpcd[1226]: eth0: adding route to 10.0.2.0/24 vm-test-run-nixbot-gitlab> gitlab # [ 16.389142] dhcpcd[1226]: eth0: adding default route via 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 16.531436] systemd[1]: Started DHCP Client. vm-test-run-nixbot-gitlab> gitlab # [ 16.536424] systemd[1]: Reached target Network is Online. vm-test-run-nixbot-gitlab> gitlab # [ 43.747773] postgres[1741]: [1741] ERROR: relation "feature_gates" does not exist at character 523 vm-test-run-nixbot-gitlab> gitlab # [ 43.748945] postgres[1741]: [1741] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), vm-test-run-nixbot-gitlab> gitlab # [ 43.750801] postgres[1741]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, vm-test-run-nixbot-gitlab> gitlab # [ 43.754824] postgres[1741]: c.collname, col_description(a.attrelid, a.attnum) AS comment, vm-test-run-nixbot-gitlab> gitlab # [ 43.758905] postgres[1741]: attidentity AS identity, vm-test-run-nixbot-gitlab> gitlab # [ 43.762978] postgres[1741]: attgenerated as attgenerated vm-test-run-nixbot-gitlab> gitlab # [ 43.766458] postgres[1741]: FROM pg_attribute a vm-test-run-nixbot-gitlab> gitlab # [ 43.768515] postgres[1741]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum vm-test-run-nixbot-gitlab> gitlab # [ 43.771874] postgres[1741]: LEFT JOIN pg_type t ON a.atttypid = t.oid vm-test-run-nixbot-gitlab> gitlab # [ 43.775911] postgres[1741]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation vm-test-run-nixbot-gitlab> gitlab # [ 43.779974] postgres[1741]: WHERE a.attrelid = '"feature_gates"'::regclass vm-test-run-nixbot-gitlab> gitlab # [ 43.783432] postgres[1741]: AND a.attnum > 0 AND NOT a.attisdropped vm-test-run-nixbot-gitlab> gitlab # [ 43.787499] postgres[1741]: ORDER BY a.attnum vm-test-run-nixbot-gitlab> gitlab # [ 43.790844] postgres[1741]: vm-test-run-nixbot-gitlab> gitlab # [ 44.019300] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1385]: W, [2026-06-29T22:05:40.925955 #1385] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 49.631393] gitlab-workhorse[1282]: time="2026-06-29T22:05:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 49.666832] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1805]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 49.668400] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1805]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 59.642887] gitlab-workhorse[1282]: time="2026-06-29T22:05:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 67.295588] postgres[1889]: [1889] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 67.296819] postgres[1889]: [1889] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 69.644342] gitlab-workhorse[1282]: time="2026-06-29T22:06:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 79.646205] gitlab-workhorse[1282]: time="2026-06-29T22:06:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 81.968937] systemd[1]: Finished gitlab-db-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 81.978298] systemd[1]: Started gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 81.981934] systemd[1]: Starting gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 82.353619] bundle[2084]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 82.362315] bundle[2084]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 82.370682] bundle[2084]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 85.318770] sidekiq-cluster[2082]: {"severity":"INFO","time":"2026-06-29T22:06:22.207Z","message":"Starting cluster with 1 processes"} vm-test-run-nixbot-gitlab> gitlab # [ 85.603727] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.487Z","pid":2084,"message":"* Enabling systemd notification integration"} vm-test-run-nixbot-gitlab> gitlab # [ 85.610549] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.509Z","pid":2084,"message":"Puma starting in cluster mode..."} vm-test-run-nixbot-gitlab> gitlab # [ 85.616700] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.510Z","pid":2084,"message":"* Puma version: 7.2.0 (\"On The Corner\")"} vm-test-run-nixbot-gitlab> gitlab # [ 85.623494] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.510Z","pid":2084,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 85.631683] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.510Z","pid":2084,"message":"* Min threads: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 85.637469] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.510Z","pid":2084,"message":"* Max threads: 4"} vm-test-run-nixbot-gitlab> gitlab # [ 85.642993] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.510Z","pid":2084,"message":"* Environment: production"} vm-test-run-nixbot-gitlab> gitlab # [ 85.649156] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.510Z","pid":2084,"message":"* Master PID: 2084"} vm-test-run-nixbot-gitlab> gitlab # [ 85.654529] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.510Z","pid":2084,"message":"* Workers: 2"} vm-test-run-nixbot-gitlab> gitlab # [ 85.658865] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.510Z","pid":2084,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} vm-test-run-nixbot-gitlab> gitlab # [ 85.664383] bundle[2084]: {"timestamp":"2026-06-29T22:06:22.510Z","pid":2084,"message":"* Preloading application"} vm-test-run-nixbot-gitlab> gitlab # [ 89.656668] gitlab-workhorse[1282]: time="2026-06-29T22:06:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 99.668416] gitlab-workhorse[1282]: time="2026-06-29T22:06:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 109.678666] gitlab-workhorse[1282]: time="2026-06-29T22:06:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 111.991992] bundle[2084]: W, [2026-06-29T22:06:48.898264 #2084] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 113.057719] sidekiq-cluster[2114]: 2026-06-29T22:06:49.964Z pid=2114 tid=tze INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2114, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>427487232} vm-test-run-nixbot-gitlab> gitlab # [ 113.258546] sidekiq-cluster[2114]: W, [2026-06-29T22:06:50.165551 #2114] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 113.300140] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:50.207Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} vm-test-run-nixbot-gitlab> gitlab # [ 115.217150] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.123Z","message":"GitLab reliable fetch activated!"} vm-test-run-nixbot-gitlab> gitlab # [ 115.217619] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.123Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 115.264239] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.171Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.281530] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.188Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.290765] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.195Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.294512] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.201Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.302713] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.209Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.309314] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.216Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.315752] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.222Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.322385] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.229Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.328547] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.235Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.333312] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.240Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.337836] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.245Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.343217] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.250Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.346167] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.253Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.349824] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.257Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.353651] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.260Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.356805] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.264Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.360658] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.267Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.365107] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.272Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.368643] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.275Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.372084] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.279Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.375717] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.283Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.379199] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.286Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.382537] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.289Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.386173] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.293Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.389524] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.296Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.392985] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.300Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.396871] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.304Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.400814] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.308Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.404193] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.311Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.407245] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.314Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.410945] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.318Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.414356] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.321Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.418210] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.325Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.421656] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.328Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.425143] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.332Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.428788] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.336Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.431708] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.339Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.435186] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.342Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.438845] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.346Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.441866] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.349Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.445335] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.352Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.448771] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.356Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.452539] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.359Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.456412] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.363Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.459622] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.366Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.463313] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.370Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.466308] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.373Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.469763] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.377Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.473663] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.380Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.477653] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.384Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.480889] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.388Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.484108] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.391Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.487681] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.394Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.499496] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.406Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.503200] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.410Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.506681] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.413Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.509961] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.417Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.513341] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.420Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.516900] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.424Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.520512] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.427Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.524209] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.431Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.527904] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.435Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.531338] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.438Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.534790] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.442Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.539352] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.446Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.572681] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.478Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.576371] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.483Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.579155] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.486Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.582347] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.489Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.586347] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.493Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.589394] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.496Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.593374] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.500Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.596400] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.503Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.599557] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.506Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.603341] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.510Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.606345] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.513Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.609562] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.516Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.613312] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.520Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.616716] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.524Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.620525] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.527Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.623561] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.530Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.627473] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.534Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.630652] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.537Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.635370] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.542Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.638554] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.545Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.642341] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.549Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.645598] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.552Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.649783] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.557Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.653127] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.560Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.656944] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.564Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.660188] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.567Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.664053] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.571Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.667299] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.574Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.670740] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.578Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.674089] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.581Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.678088] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.585Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.681705] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.589Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.685606] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.592Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.689186] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.596Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.700820] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.608Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.703656] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.610Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.706962] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.614Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.711989] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.619Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.715769] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.623Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.718897] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.626Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.723192] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.630Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.725631] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.632Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.729156] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.636Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.732866] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.640Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 115.736500] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:06:52.643Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 119.680461] gitlab-workhorse[1282]: time="2026-06-29T22:06:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 129.683230] gitlab-workhorse[1282]: time="2026-06-29T22:07:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 139.687877] gitlab-workhorse[1282]: time="2026-06-29T22:07:16Z" level=error correlation_id= duration_ms=2 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 139.872393] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:16.778Z","message":"Booted Rails 7.2.3 application in production environment"} vm-test-run-nixbot-gitlab> gitlab # [ 139.873766] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:16.780Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 139.875523] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:16.780Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} vm-test-run-nixbot-gitlab> gitlab # [ 139.877191] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:16.780Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} vm-test-run-nixbot-gitlab> gitlab # [ 139.887239] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:16.794Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 141.711408] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:18.618Z","message":"Cleaning working queues"} vm-test-run-nixbot-gitlab> gitlab # [ 141.715102] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:18.622Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7f4a99d42a6a964a0434f4ec","created_at":"2026-06-29T22:06:18.257Z","trace_propagation_headers":{"sentry-trace":"c00dd66fbdba43f18817e5ed90080496-4b2fcfd59e55411b","baggage":"sentry-trace_id=c00dd66fbdba43f18817e5ed90080496,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"29bce411bb1f0d3e82e97eaf2631eac3","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T22:06:18.267Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7f4a99d42a6a964a0434f4ec: start","job_status":"start","queue_duration_s":60.354752,"scheduling_latency_s":60.354819} vm-test-run-nixbot-gitlab> gitlab # [ 141.811799] bundle[2084]: {"timestamp":"2026-06-29T22:07:18.718Z","pid":2084,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} vm-test-run-nixbot-gitlab> gitlab # [ 141.812191] bundle[2084]: {"timestamp":"2026-06-29T22:07:18.718Z","pid":2084,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} vm-test-run-nixbot-gitlab> gitlab # [ 141.814320] bundle[2084]: {"timestamp":"2026-06-29T22:07:18.718Z","pid":2084,"message":"! #\u003cThread:0x0000742c89744be8 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 141.816411] bundle[2084]: {"timestamp":"2026-06-29T22:07:18.718Z","pid":2084,"message":"! #\u003cThread:0x0000742c899a4dc0 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 141.818613] bundle[2084]: {"timestamp":"2026-06-29T22:07:18.719Z","pid":2084,"message":"Use Ctrl-C to stop"} vm-test-run-nixbot-gitlab> gitlab # [ 141.843315] bundle[2084]: {"timestamp":"2026-06-29T22:07:18.750Z","pid":2084,"message":"Sending status to systemd every 1.0 sec"} vm-test-run-nixbot-gitlab> gitlab # [ 141.895085] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:18.802Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7f4a99d42a6a964a0434f4ec","created_at":"2026-06-29T22:06:18.257Z","trace_propagation_headers":{"sentry-trace":"c00dd66fbdba43f18817e5ed90080496-4b2fcfd59e55411b","baggage":"sentry-trace_id=c00dd66fbdba43f18817e5ed90080496,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"29bce411bb1f0d3e82e97eaf2631eac3","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T22:06:18.267Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7f4a99d42a6a964a0434f4ec: done: 0.180253 sec","job_status":"done","queue_duration_s":60.354752,"scheduling_latency_s":60.354819,"redis_calls":26,"redis_duration_s":0.009850000000000001,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.00655,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002009,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001291,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08170814900000001,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.180253,"completed_at":"2026-06-29T22:07:18.801Z","load_balancing_strategy":"primary","db_duration_s":0.057692,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 142.203150] bundle[2084]: {"timestamp":"2026-06-29T22:07:19.109Z","pid":2084,"message":"- Worker 1 (PID: 2673) booted in 0.3s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 142.256381] bundle[2084]: {"timestamp":"2026-06-29T22:07:19.163Z","pid":2084,"message":"- Worker 0 (PID: 2669) booted in 0.38s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 142.266239] systemd[1]: Started gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 142.268054] systemd[1]: Reached target Common target for all GitLab services.. vm-test-run-nixbot-gitlab> gitlab # [ 142.272639] systemd[1]: Starting setup-gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 142.299153] setup-gitlab-start[2697]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' vm-test-run-nixbot-gitlab> gitlab # [ 144.777381] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:07:21 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 2429 vm-test-run-nixbot-gitlab> gitlab # [ 144.785708] setup-gitlab-start[2716]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token vm-test-run-nixbot-gitlab> gitlab # [ 144.787919] setup-gitlab-start[2717]: ++ jq -r .access_token vm-test-run-nixbot-gitlab> gitlab # [ 145.577130] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:07:22 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 770 vm-test-run-nixbot-gitlab> gitlab # [ 145.583746] setup-gitlab-start[2697]: + BEARER=4a2bb711b1be83cc7d1cf0e2d21072cc2e867617e138b17e83dcea2d5d241fac vm-test-run-nixbot-gitlab> gitlab # [ 145.585135] setup-gitlab-start[2697]: + AUTH='Authorization: Bearer 4a2bb711b1be83cc7d1cf0e2d21072cc2e867617e138b17e83dcea2d5d241fac' vm-test-run-nixbot-gitlab> gitlab # [ 145.586612] setup-gitlab-start[2697]: + curl -fs -X PUT -H 'Authorization: Bearer 4a2bb711b1be83cc7d1cf0e2d21072cc2e867617e138b17e83dcea2d5d241fac' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true' vm-test-run-nixbot-gitlab> gitlab # [ 146.358619] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:07:23 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 754 vm-test-run-nixbot-gitlab> gitlab # [ 146.368969] setup-gitlab-start[2697]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear vm-test-run-nixbot-gitlab> gitlab # [ 146.387112] runuser[2735]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) vm-test-run-nixbot-gitlab> gitlab # [ 146.773637] setup-gitlab-start[2736]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 146.774356] setup-gitlab-start[2736]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 146.777206] setup-gitlab-start[2736]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 147.367837] setup-gitlab-start[2736]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 147.370187] setup-gitlab-start[2736]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 147.372130] setup-gitlab-start[2736]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 153.473935] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:30.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"26df50c2481cdfa954ee9bbf","created_at":"2026-06-29T22:07:30.338Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a65195d3e7a6fd8b41d4710d1d9b9044","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.376Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-26df50c2481cdfa954ee9bbf: start","job_status":"start","queue_duration_s":0.003354,"scheduling_latency_s":0.003522} vm-test-run-nixbot-gitlab> gitlab # [ 153.639152] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:30.545Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"26df50c2481cdfa954ee9bbf","created_at":"2026-06-29T22:07:30.338Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a65195d3e7a6fd8b41d4710d1d9b9044","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.376Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-26df50c2481cdfa954ee9bbf: done: 0.16599 sec","job_status":"done","queue_duration_s":0.003354,"scheduling_latency_s":0.003522,"redis_calls":12,"redis_duration_s":0.02178,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.006134,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007485,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001376152,"duration_s":0.16599,"completed_at":"2026-06-29T22:07:30.545Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 153.741233] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:30.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"5864f380529b20199bcfea2a","created_at":"2026-06-29T22:07:30.444Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52e0f5d8e2f1fb2bd7b0cf9f683b9ad3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.469Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-5864f380529b20199bcfea2a: start","job_status":"start","queue_duration_s":0.177882,"scheduling_latency_s":0.177955} vm-test-run-nixbot-gitlab> gitlab # [ 153.995490] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:30.901Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"5864f380529b20199bcfea2a","created_at":"2026-06-29T22:07:30.444Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52e0f5d8e2f1fb2bd7b0cf9f683b9ad3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.469Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-5864f380529b20199bcfea2a: done: 0.253934 sec","job_status":"done","queue_duration_s":0.177882,"scheduling_latency_s":0.177955,"redis_calls":12,"redis_duration_s":0.05374,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003879,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.044352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005509,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.137488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001842412,"exclusive_lock_hold_duration_s":0.151989429,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.253934,"completed_at":"2026-06-29T22:07:30.901Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036247,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null} vm-test-run-nixbot-gitlab> gitlab # [ 154.099768] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:30.908Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"121e19a566e36a55b1ab9030","created_at":"2026-06-29T22:07:30.499Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"188df8c5f6f883c36e171f9b5701d120","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.526Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-121e19a566e36a55b1ab9030: start","job_status":"start","queue_duration_s":0.382022,"scheduling_latency_s":0.382083} vm-test-run-nixbot-gitlab> gitlab # [ 154.740407] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:31.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"121e19a566e36a55b1ab9030","created_at":"2026-06-29T22:07:30.499Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"188df8c5f6f883c36e171f9b5701d120","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.526Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-121e19a566e36a55b1ab9030: done: 0.737836 sec","job_status":"done","queue_duration_s":0.382022,"scheduling_latency_s":0.382083,"redis_calls":14,"redis_duration_s":0.00893,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002293,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003397,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.118,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.069,"db_ci_duration_s":0.049,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.356442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002031822,"exclusive_lock_hold_duration_s":0.624961552,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.737836,"completed_at":"2026-06-29T22:07:31.645Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.154152,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 154.811292] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:31.718Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3607df4eae0961d20113965c","created_at":"2026-06-29T22:07:30.636Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd8b83ba896a48a6bca5fa9e93d85865","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.656Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3607df4eae0961d20113965c: start","job_status":"start","queue_duration_s":1.061115,"scheduling_latency_s":1.061156} vm-test-run-nixbot-gitlab> gitlab # [ 154.852814] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:31.759Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 154.870550] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:31.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3607df4eae0961d20113965c","created_at":"2026-06-29T22:07:30.636Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd8b83ba896a48a6bca5fa9e93d85865","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.656Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3607df4eae0961d20113965c: done: 0.05893 sec","job_status":"done","queue_duration_s":1.061115,"scheduling_latency_s":1.061156,"redis_calls":10,"redis_duration_s":0.005478,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001728,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002823,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000750095,"duration_s":0.05893,"completed_at":"2026-06-29T22:07:31.776Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000796,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 154.941172] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:31.847Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"39a52e20e376526c2a86aeaa","created_at":"2026-06-29T22:07:30.708Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef2b4ad14f108a7467a06dae5f86118e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T22:07:30.735Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-39a52e20e376526c2a86aeaa: start","job_status":"start","queue_duration_s":1.111791,"scheduling_latency_s":1.111865} vm-test-run-nixbot-gitlab> gitlab # [ 154.989268] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:31.896Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 155.015253] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:31.921Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"39a52e20e376526c2a86aeaa","created_at":"2026-06-29T22:07:30.708Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef2b4ad14f108a7467a06dae5f86118e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T22:07:30.735Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-39a52e20e376526c2a86aeaa: done: 0.074033 sec","job_status":"done","queue_duration_s":1.111791,"scheduling_latency_s":1.111865,"redis_calls":10,"redis_duration_s":0.013077,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001851,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005347,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005879,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001184787,"duration_s":0.074033,"completed_at":"2026-06-29T22:07:31.921Z","load_balancing_strategy":"primary","db_duration_s":0.001233,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 155.083664] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:31.990Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b2b358551e1d9f154e9dd4b5","created_at":"2026-06-29T22:07:30.757Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2444b2213d06dbd3fd57e41e7e3ae0ca","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.784Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b2b358551e1d9f154e9dd4b5: start","job_status":"start","queue_duration_s":1.205533,"scheduling_latency_s":1.205621} vm-test-run-nixbot-gitlab> gitlab # [ 155.180871] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b2b358551e1d9f154e9dd4b5","created_at":"2026-06-29T22:07:30.757Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2444b2213d06dbd3fd57e41e7e3ae0ca","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.784Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b2b358551e1d9f154e9dd4b5: done: 0.097115 sec","job_status":"done","queue_duration_s":1.205533,"scheduling_latency_s":1.205621,"redis_calls":12,"redis_duration_s":0.008911,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003654,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001348495,"exclusive_lock_hold_duration_s":0.024617375,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.097115,"completed_at":"2026-06-29T22:07:32.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 155.274293] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.180Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bf192946c80e7057f58f251b","created_at":"2026-06-29T22:07:30.889Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"045f35685a741040d033ef3f3e177541","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.992Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bf192946c80e7057f58f251b: start","job_status":"start","queue_duration_s":1.18829,"scheduling_latency_s":1.18833} vm-test-run-nixbot-gitlab> gitlab # [ 155.355586] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bf192946c80e7057f58f251b","created_at":"2026-06-29T22:07:30.889Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"045f35685a741040d033ef3f3e177541","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:30.992Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bf192946c80e7057f58f251b: done: 0.08095 sec","job_status":"done","queue_duration_s":1.18829,"scheduling_latency_s":1.18833,"redis_calls":10,"redis_duration_s":0.005242,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00087,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001140368,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.08095,"completed_at":"2026-06-29T22:07:32.261Z","load_balancing_strategy":"primary","db_duration_s":0.020181,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 155.361347] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0fee5099e9bfbf681eca1e1a","created_at":"2026-06-29T22:07:30.998Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e837f7b61012bbe96667ff57245a735","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.005Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0fee5099e9bfbf681eca1e1a: start","job_status":"start","queue_duration_s":1.261052,"scheduling_latency_s":1.261094} vm-test-run-nixbot-gitlab> gitlab # [ 155.393185] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.300Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 155.412226] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0fee5099e9bfbf681eca1e1a","created_at":"2026-06-29T22:07:30.998Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e837f7b61012bbe96667ff57245a735","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.005Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0fee5099e9bfbf681eca1e1a: done: 0.052107 sec","job_status":"done","queue_duration_s":1.261052,"scheduling_latency_s":1.261094,"redis_calls":10,"redis_duration_s":0.007018,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001218,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003803,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000974985,"duration_s":0.052107,"completed_at":"2026-06-29T22:07:32.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000833,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 155.487407] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.325Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3e48a90fd46aec486ccd802b","created_at":"2026-06-29T22:07:31.018Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f34f996ec4a72f52b41907d2887913b2","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.044Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3e48a90fd46aec486ccd802b: start","job_status":"start","queue_duration_s":1.281102,"scheduling_latency_s":1.281131} vm-test-run-nixbot-gitlab> gitlab # [ 155.910670] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.817Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3e48a90fd46aec486ccd802b","created_at":"2026-06-29T22:07:31.018Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f34f996ec4a72f52b41907d2887913b2","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.044Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3e48a90fd46aec486ccd802b: done: 0.492015 sec","job_status":"done","queue_duration_s":1.281102,"scheduling_latency_s":1.281131,"redis_calls":574,"redis_duration_s":0.168182,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0009,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002048,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.165234,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.366564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000427988,"duration_s":0.492015,"completed_at":"2026-06-29T22:07:32.817Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 155.912489] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.819Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ae6954a124d152d4a951c334","created_at":"2026-06-29T22:07:31.077Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.090Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ae6954a124d152d4a951c334: start","job_status":"start","queue_duration_s":1.728946,"scheduling_latency_s":1.728998} vm-test-run-nixbot-gitlab> gitlab # [ 156.022590] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.924Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ae6954a124d152d4a951c334","created_at":"2026-06-29T22:07:31.077Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.090Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ae6954a124d152d4a951c334: done: 0.105541 sec","job_status":"done","queue_duration_s":1.728946,"scheduling_latency_s":1.728998,"redis_calls":71,"redis_duration_s":0.020878,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002504,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001384,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.016155,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000835,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000493917,"duration_s":0.105541,"completed_at":"2026-06-29T22:07:32.924Z","load_balancing_strategy":"primary","db_duration_s":0.002031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.037183] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f387464add8d0ebe0be28203","created_at":"2026-06-29T22:07:31.101Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5cedc2eddac1cf061807fd60cacdba3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.114Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f387464add8d0ebe0be28203: start","job_status":"start","queue_duration_s":1.812277,"scheduling_latency_s":1.812302} vm-test-run-nixbot-gitlab> gitlab # [ 156.110475] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.952Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f387464add8d0ebe0be28203","created_at":"2026-06-29T22:07:31.101Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5cedc2eddac1cf061807fd60cacdba3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.114Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f387464add8d0ebe0be28203: done: 0.025895 sec","job_status":"done","queue_duration_s":1.812277,"scheduling_latency_s":1.812302,"redis_calls":10,"redis_duration_s":0.002377,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000459,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001453,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000351162,"duration_s":0.025895,"completed_at":"2026-06-29T22:07:32.952Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004648,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.128291] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1568e23fe5ee7ec1a9b6346d","created_at":"2026-06-29T22:07:31.121Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac9a3ec2085cc6f02d6117cedf37725f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T22:07:31.131Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1568e23fe5ee7ec1a9b6346d: start","job_status":"start","queue_duration_s":1.823067,"scheduling_latency_s":1.82309} vm-test-run-nixbot-gitlab> gitlab # [ 156.149763] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.965Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 156.155741] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1568e23fe5ee7ec1a9b6346d","created_at":"2026-06-29T22:07:31.121Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac9a3ec2085cc6f02d6117cedf37725f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T22:07:31.131Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1568e23fe5ee7ec1a9b6346d: done: 0.017441 sec","job_status":"done","queue_duration_s":1.823067,"scheduling_latency_s":1.82309,"redis_calls":10,"redis_duration_s":0.002377,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000473,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001409,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000360101,"duration_s":0.017441,"completed_at":"2026-06-29T22:07:32.971Z","load_balancing_strategy":"primary","db_duration_s":0.000406,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.229739] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:32.973Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d3a00253256c5874fa258fb9","created_at":"2026-06-29T22:07:31.136Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7080f9d56419711e4675cce065d890c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T22:07:31.142Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d3a00253256c5874fa258fb9: start","job_status":"start","queue_duration_s":1.83015,"scheduling_latency_s":1.830171} vm-test-run-nixbot-gitlab> gitlab # [ 156.243416] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d3a00253256c5874fa258fb9","created_at":"2026-06-29T22:07:31.136Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7080f9d56419711e4675cce065d890c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T22:07:31.142Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d3a00253256c5874fa258fb9: done: 0.029872 sec","job_status":"done","queue_duration_s":1.83015,"scheduling_latency_s":1.830171,"redis_calls":10,"redis_duration_s":0.00248,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001545,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000512356,"duration_s":0.029872,"completed_at":"2026-06-29T22:07:33.002Z","load_balancing_strategy":"primary","db_duration_s":0.00493,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.347613] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.004Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2926d395895db1eec641ed6d","created_at":"2026-06-29T22:07:31.146Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fe7c0c4c8d06312afd07ff7a86fba3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.153Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2926d395895db1eec641ed6d: start","job_status":"start","queue_duration_s":1.851479,"scheduling_latency_s":1.851501} vm-test-run-nixbot-gitlab> gitlab # [ 156.351447] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.023Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2926d395895db1eec641ed6d","created_at":"2026-06-29T22:07:31.146Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fe7c0c4c8d06312afd07ff7a86fba3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.153Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2926d395895db1eec641ed6d: done: 0.018447 sec","job_status":"done","queue_duration_s":1.851479,"scheduling_latency_s":1.851501,"redis_calls":10,"redis_duration_s":0.0024029999999999998,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349207,"duration_s":0.018447,"completed_at":"2026-06-29T22:07:33.023Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.440578] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"76931b8261fc94488f7d48b0","created_at":"2026-06-29T22:07:31.167Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65525c1fdc8dcd556ba0b2f62e54c391","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.177Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-76931b8261fc94488f7d48b0: start","job_status":"start","queue_duration_s":1.846939,"scheduling_latency_s":1.846961} vm-test-run-nixbot-gitlab> gitlab # [ 156.442446] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.048Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"76931b8261fc94488f7d48b0","created_at":"2026-06-29T22:07:31.167Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65525c1fdc8dcd556ba0b2f62e54c391","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.177Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-76931b8261fc94488f7d48b0: done: 0.02447 sec","job_status":"done","queue_duration_s":1.846939,"scheduling_latency_s":1.846961,"redis_calls":12,"redis_duration_s":0.003158,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000528,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000509282,"exclusive_lock_hold_duration_s":0.005934274,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.02447,"completed_at":"2026-06-29T22:07:33.048Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.531143] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.050Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"eb86fddc9e4b3ad7b1cdb786","created_at":"2026-06-29T22:07:31.185Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a21da8df6775a0650383e38310260c6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.194Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-eb86fddc9e4b3ad7b1cdb786: start","job_status":"start","queue_duration_s":1.855862,"scheduling_latency_s":1.855883} vm-test-run-nixbot-gitlab> gitlab # [ 156.532715] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.080Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"eb86fddc9e4b3ad7b1cdb786","created_at":"2026-06-29T22:07:31.185Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a21da8df6775a0650383e38310260c6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.194Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-eb86fddc9e4b3ad7b1cdb786: done: 0.029704 sec","job_status":"done","queue_duration_s":1.855862,"scheduling_latency_s":1.855883,"redis_calls":16,"redis_duration_s":0.0038599999999999997,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001951,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001455,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040061,"duration_s":0.029704,"completed_at":"2026-06-29T22:07:33.080Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.616574] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ExpireBuildArtifactsWorker","jid":"7e668b029aba348583ac0c1a","created_at":"2026-06-29T22:07:31.201Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63c1c3103fc39317da1e4ddfd61884eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:7be9ea93da0bda6381b334cf2932692b41842a57ea0aeac11899369225b9bb7e","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.207Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ExpireBuildArtifactsWorker JID-7e668b029aba348583ac0c1a: start","job_status":"start","queue_duration_s":1.874528,"scheduling_latency_s":1.87455} vm-test-run-nixbot-gitlab> gitlab # [ 156.618371] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ExpireBuildArtifactsWorker","jid":"7e668b029aba348583ac0c1a","created_at":"2026-06-29T22:07:31.201Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63c1c3103fc39317da1e4ddfd61884eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:7be9ea93da0bda6381b334cf2932692b41842a57ea0aeac11899369225b9bb7e","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.207Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ExpireBuildArtifactsWorker JID-7e668b029aba348583ac0c1a: done: 0.018129 sec","job_status":"done","queue_duration_s":1.874528,"scheduling_latency_s":1.87455,"redis_calls":10,"redis_duration_s":0.002354,"redis_read_bytes":9,"redis_write_bytes":1035,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00045,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001454,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00045,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000359264,"duration_s":0.018129,"completed_at":"2026-06-29T22:07:33.100Z","load_balancing_strategy":"primary","db_duration_s":0.000557,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.689276] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7250f3a2e5a8feeed5d818d3","created_at":"2026-06-29T22:07:31.236Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0154218b1141633f073b89815c5ea852","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.244Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7250f3a2e5a8feeed5d818d3: start","job_status":"start","queue_duration_s":1.857064,"scheduling_latency_s":1.857094} vm-test-run-nixbot-gitlab> gitlab # [ 156.693171] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.127Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7250f3a2e5a8feeed5d818d3","created_at":"2026-06-29T22:07:31.236Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0154218b1141633f073b89815c5ea852","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.244Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7250f3a2e5a8feeed5d818d3: done: 0.025792 sec","job_status":"done","queue_duration_s":1.857064,"scheduling_latency_s":1.857094,"redis_calls":12,"redis_duration_s":0.003026,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000526,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001382,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000520178,"exclusive_lock_hold_duration_s":0.007177448,"duration_s":0.025792,"completed_at":"2026-06-29T22:07:33.127Z","load_balancing_strategy":"primary","db_duration_s":0.004529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.774404] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"095c17cecaacd2de433fc23b","created_at":"2026-06-29T22:07:31.248Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d38a2dab2ca16b53fb384ae28c9763a5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.253Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-095c17cecaacd2de433fc23b: start","job_status":"start","queue_duration_s":1.875496,"scheduling_latency_s":1.875518} vm-test-run-nixbot-gitlab> gitlab # [ 156.782467] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.142Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 156.787970] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"095c17cecaacd2de433fc23b","created_at":"2026-06-29T22:07:31.248Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d38a2dab2ca16b53fb384ae28c9763a5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.253Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-095c17cecaacd2de433fc23b: done: 0.019804 sec","job_status":"done","queue_duration_s":1.875496,"scheduling_latency_s":1.875518,"redis_calls":10,"redis_duration_s":0.002422,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0005,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000397258,"duration_s":0.019804,"completed_at":"2026-06-29T22:07:33.148Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002228,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.898859] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4b5e3c75c741e97377a76b1f","created_at":"2026-06-29T22:07:31.258Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.263Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4b5e3c75c741e97377a76b1f: start","job_status":"start","queue_duration_s":1.886442,"scheduling_latency_s":1.886463} vm-test-run-nixbot-gitlab> gitlab # [ 156.900332] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4b5e3c75c741e97377a76b1f","created_at":"2026-06-29T22:07:31.258Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.263Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4b5e3c75c741e97377a76b1f: done: 0.031963 sec","job_status":"done","queue_duration_s":1.886442,"scheduling_latency_s":1.886463,"redis_calls":19,"redis_duration_s":0.004593,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000929,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000315,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00242,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000353676,"duration_s":0.031963,"completed_at":"2026-06-29T22:07:33.181Z","load_balancing_strategy":"primary","db_duration_s":0.003979,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 156.972472] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"7461c97945d4cdebc406a822","created_at":"2026-06-29T22:07:31.278Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e238ce52dd001927b716339c3a78679","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.284Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-7461c97945d4cdebc406a822: start","job_status":"start","queue_duration_s":1.899128,"scheduling_latency_s":1.899149} vm-test-run-nixbot-gitlab> gitlab # [ 156.974068] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.218Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"7461c97945d4cdebc406a822","created_at":"2026-06-29T22:07:31.278Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e238ce52dd001927b716339c3a78679","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.284Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-7461c97945d4cdebc406a822: done: 0.034788 sec","job_status":"done","queue_duration_s":1.899128,"scheduling_latency_s":1.899149,"redis_calls":12,"redis_duration_s":0.00318,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000581,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001104,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000589181,"exclusive_lock_hold_duration_s":0.015626847,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.034788,"completed_at":"2026-06-29T22:07:33.218Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008155,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.044149] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0423def8ac31e81c1458b7eb","created_at":"2026-06-29T22:07:31.288Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ec72de0fe301b30ffc42d777726fb13","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.294Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0423def8ac31e81c1458b7eb: start","job_status":"start","queue_duration_s":1.924757,"scheduling_latency_s":1.924778} vm-test-run-nixbot-gitlab> gitlab # [ 157.113987] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.231Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 157.115262] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.237Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0423def8ac31e81c1458b7eb","created_at":"2026-06-29T22:07:31.288Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ec72de0fe301b30ffc42d777726fb13","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.294Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0423def8ac31e81c1458b7eb: done: 0.018078 sec","job_status":"done","queue_duration_s":1.924757,"scheduling_latency_s":1.924778,"redis_calls":10,"redis_duration_s":0.002335,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001411,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036094,"duration_s":0.018078,"completed_at":"2026-06-29T22:07:33.237Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000632,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.186141] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.239Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"85ca014276f5063afcf01ca9","created_at":"2026-06-29T22:07:31.299Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58e523c5747101c4afd57bf55bd767e4","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.306Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-85ca014276f5063afcf01ca9: start","job_status":"start","queue_duration_s":1.933085,"scheduling_latency_s":1.933106} vm-test-run-nixbot-gitlab> gitlab # [ 157.187312] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"85ca014276f5063afcf01ca9","created_at":"2026-06-29T22:07:31.299Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58e523c5747101c4afd57bf55bd767e4","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T22:07:31.306Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-85ca014276f5063afcf01ca9: done: 0.04192 sec","job_status":"done","queue_duration_s":1.933085,"scheduling_latency_s":1.933106,"redis_calls":13,"redis_duration_s":0.008014,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000898,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002537,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004579,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.016027735,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.04192,"completed_at":"2026-06-29T22:07:33.281Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011526,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 157.258243] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"777f6d8ed712c38640fbd0f5","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-777f6d8ed712c38640fbd0f5: start","job_status":"start","queue_duration_s":0.368347,"scheduling_latency_s":0.368369} vm-test-run-nixbot-gitlab> gitlab # [ 157.259852] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"777f6d8ed712c38640fbd0f5","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sideki[ 157.597123] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1065998028 wd_nsec: 1065997534 vm-test-run-nixbot-gitlab> gitlab # q.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-777f6d8ed712c38640fbd0f5: done: 0.023406 sec","job_status":"done","queue_duration_s":0.368347,"scheduling_latency_s":0.368369,"redis_calls":16,"redis_duration_s":0.00526,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002012,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002754,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000705397,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023406,"completed_at":"2026-06-29T22:07:33.306Z","load_balancing_strategy":"primary","db_duration_s":0.000673,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.386406] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95bb860c8e9f8d6d68931c4a","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95bb860c8e9f8d6d68931c4a: start","job_status":"start","queue_duration_s":0.393998,"scheduling_latency_s":0.394019} vm-test-run-nixbot-gitlab> gitlab # [ 157.387108] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95bb860c8e9f8d6d68931c4a","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95bb860c8e9f8d6d68931c4a: done: 0.019364 sec","job_status":"done","queue_duration_s":0.393998,"scheduling_latency_s":0.394019,"redis_calls":13,"redis_duration_s":0.005749,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019364,"completed_at":"2026-06-29T22:07:33.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.459273] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b9e70fa20f78e5b9f74773e","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b9e70fa20f78e5b9f74773e: start","job_status":"start","queue_duration_s":0.415469,"scheduling_latency_s":0.415492} vm-test-run-nixbot-gitlab> gitlab # [ 157.461092] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b9e70fa20f78e5b9f74773e","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b9e70fa20f78e5b9f74773e: done: 0.019617 sec","job_status":"done","queue_duration_s":0.415469,"scheduling_latency_s":0.415492,"redis_calls":13,"redis_duration_s":0.006316,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002368,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019617,"completed_at":"2026-06-29T22:07:33.349Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.478339] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"638ff9d2a2124e1d298faa06","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-638ff9d2a2124e1d298faa06: start","job_status":"start","queue_duration_s":0.436559,"scheduling_latency_s":0.436583} vm-test-run-nixbot-gitlab> gitlab # [ 157.558902] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"638ff9d2a2124e1d298faa06","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-638ff9d2a2124e1d298faa06: done: 0.016035 sec","job_status":"done","queue_duration_s":0.436559,"scheduling_latency_s":0.436583,"redis_calls":13,"redis_duration_s":0.003287,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002281,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016035,"completed_at":"2026-06-29T22:07:33.367Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.561065] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c82064ecc73a5c63f00b483","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c82064ecc73a5c63f00b483: start","job_status":"start","queue_duration_s":0.454269,"scheduling_latency_s":0.454292} vm-test-run-nixbot-gitlab> gitlab # [ 157.579378] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c82064ecc73a5c63f00b483","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c82064ecc73a5c63f00b483: done: 0.01615 sec","job_status":"done","queue_duration_s":0.454269,"scheduling_latency_s":0.454292,"redis_calls":13,"redis_duration_s":0.003179,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01615,"completed_at":"2026-06-29T22:07:33.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.659116] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67bfce05cda8966e38f4b12e","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67bfce05cda8966e38f4b12e: start","job_status":"start","queue_duration_s":0.472358,"scheduling_latency_s":0.472406} vm-test-run-nixbot-gitlab> gitlab # [ 157.660859] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67bfce05cda8966e38f4b12e","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67bfce05cda8966e38f4b12e: done: 0.021243 sec","job_status":"done","queue_duration_s":0.472358,"scheduling_latency_s":0.472406,"redis_calls":13,"redis_duration_s":0.004005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001426,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002579,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021243,"completed_at":"2026-06-29T22:07:33.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.742971] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b24f047b596c8385e021024e","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b24f047b596c8385e021024e: start","job_status":"start","queue_duration_s":0.494856,"scheduling_latency_s":0.494878} vm-test-run-nixbot-gitlab> gitlab # [ 157.744382] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b24f047b596c8385e021024e","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b24f047b596c8385e021024e: done: 0.015859 sec","job_status":"done","queue_duration_s":0.494856,"scheduling_latency_s":0.494878,"redis_calls":13,"redis_duration_s":0.003193,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015859,"completed_at":"2026-06-29T22:07:33.425Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.813844] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc1501078beccc60118fd91a","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc1501078beccc60118fd91a: start","job_status":"start","queue_duration_s":0.512067,"scheduling_latency_s":0.512089} vm-test-run-nixbot-gitlab> gitlab # [ 157.885250] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc1501078beccc60118fd91a","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc1501078beccc60118fd91a: done: 0.015688 sec","job_status":"done","queue_duration_s":0.512067,"scheduling_latency_s":0.512089,"redis_calls":13,"redis_duration_s":0.003016,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015688,"completed_at":"2026-06-29T22:07:33.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.887146] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52c958f991154597b7eff5e2","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52c958f991154597b7eff5e2: start","job_status":"start","queue_duration_s":0.529242,"scheduling_latency_s":0.529265} vm-test-run-nixbot-gitlab> gitlab # [ 157.904400] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52c958f991154597b7eff5e2","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52c958f991154597b7eff5e2: done: 0.015641 sec","job_status":"done","queue_duration_s":0.529242,"scheduling_latency_s":0.529265,"redis_calls":13,"redis_duration_s":0.002972,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015641,"completed_at":"2026-06-29T22:07:33.459Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 157.974546] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e048fa8aeca6befbc27faec","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e048fa8aeca6befbc27faec: start","job_status":"start","queue_duration_s":0.546389,"scheduling_latency_s":0.54641} vm-test-run-nixbot-gitlab> gitlab # [ 158.046391] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e048fa8aeca6befbc27faec","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e048fa8aeca6befbc27faec: done: 0.015745 sec","job_status":"done","queue_duration_s":0.546389,"scheduling_latency_s":0.54641,"redis_calls":13,"redis_duration_s":0.003182,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015745,"completed_at":"2026-06-29T22:07:33.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.048352] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0af70035be6c4fa897701cbb","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0af70035be6c4fa897701cbb: start","job_status":"start","queue_duration_s":0.563562,"scheduling_latency_s":0.563583} vm-test-run-nixbot-gitlab> gitlab # [ 158.067988] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0af70035be6c4fa897701cbb","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0af70035be6c4fa897701cbb: done: 0.017372 sec","job_status":"done","queue_duration_s":0.563562,"scheduling_latency_s":0.563583,"redis_calls":13,"redis_duration_s":0.004762,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001397,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003365,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017372,"completed_at":"2026-06-29T22:07:33.495Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.137388] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"262f762481565fea351847b2","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-262f762481565fea351847b2: start","job_status":"start","queue_duration_s":0.582762,"scheduling_latency_s":0.582784} vm-test-run-nixbot-gitlab> gitlab # [ 158.149400] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"262f762481565fea351847b2","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-262f762481565fea351847b2: done: 0.017504 sec","job_status":"done","queue_duration_s":0.582762,"scheduling_latency_s":0.582784,"redis_calls":13,"redis_duration_s":0.004345,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001291,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017504,"completed_at":"2026-06-29T22:07:33.514Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.219270] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2420df878001f35e01543fce","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2420df878001f35e01543fce: start","job_status":"start","queue_duration_s":0.602043,"scheduling_latency_s":0.602066} vm-test-run-nixbot-gitlab> gitlab # [ 158.230402] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2420df878001f35e01543fce","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2420df878001f35e01543fce: done: 0.01969 sec","job_status":"done","queue_duration_s":0.602043,"scheduling_latency_s":0.602066,"redis_calls":13,"redis_duration_s":0.006684,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003799,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002885,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01969,"completed_at":"2026-06-29T22:07:33.536Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.299478] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9b5db301582bb037f0ede16","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9b5db301582bb037f0ede16: start","job_status":"start","queue_duration_s":0.623587,"scheduling_latency_s":0.623608} vm-test-run-nixbot-gitlab> gitlab # [ 158.373523] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9b5db301582bb037f0ede16","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9b5db301582bb037f0ede16: done: 0.017204 sec","job_status":"done","queue_duration_s":0.623587,"scheduling_latency_s":0.623608,"redis_calls":13,"redis_duration_s":0.004424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017204,"completed_at":"2026-06-29T22:07:33.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.375182] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12303045ff9df98d633b18fa","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12303045ff9df98d633b18fa: start","job_status":"start","queue_duration_s":0.64263,"scheduling_latency_s":0.642651} vm-test-run-nixbot-gitlab> gitlab # [ 158.444057] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12303045ff9df98d633b18fa","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12303045ff9df98d633b18fa: done: 0.017406 sec","job_status":"done","queue_duration_s":0.64263,"scheduling_latency_s":0.642651,"redis_calls":13,"redis_duration_s":0.004554,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001399,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003155,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017406,"completed_at":"2026-06-29T22:07:33.574Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.454916] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80f1ceb41d841bc236e67bb9","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80f1ceb41d841bc236e67bb9: start","job_status":"start","queue_duration_s":0.661825,"scheduling_latency_s":0.661847} vm-test-run-nixbot-gitlab> gitlab # [ 158.541270] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80f1ceb41d841bc236e67bb9","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80f1ceb41d841bc236e67bb9: done: 0.017289 sec","job_status":"done","queue_duration_s":0.661825,"scheduling_latency_s":0.661847,"redis_calls":13,"redis_duration_s":0.004483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017289,"completed_at":"2026-06-29T22:07:33.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.543294] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80f60d6aaff1a2912e46c48d","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80f60d6aaff1a2912e46c48d: start","job_status":"start","queue_duration_s":0.684647,"scheduling_latency_s":0.684671} vm-test-run-nixbot-gitlab> gitlab # [ 158.615502] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80f60d6aaff1a2912e46c48d","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80f60d6aaff1a2912e46c48d: done: 0.019487 sec","job_status":"done","queue_duration_s":0.684647,"scheduling_latency_s":0.684671,"redis_calls":13,"redis_duration_s":0.006312999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002271,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019487,"completed_at":"2026-06-29T22:07:33.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.617234] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b99a25d4a74d8045b7bfb1a0","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b99a25d4a74d8045b7bfb1a0: start","job_status":"start","queue_duration_s":0.705405,"scheduling_latency_s":0.705427} vm-test-run-nixbot-gitlab> gitlab # [ 158.637428] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b99a25d4a74d8045b7bfb1a0","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b99a25d4a74d8045b7bfb1a0: done: 0.015451 sec","job_status":"done","queue_duration_s":0.705405,"scheduling_latency_s":0.705427,"redis_calls":13,"redis_duration_s":0.002827,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015451,"completed_at":"2026-06-29T22:07:33.635Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.740309] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"284cf16a0b7a50f607689b5e","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-284cf16a0b7a50f607689b5e: start","job_status":"start","queue_duration_s":0.722313,"scheduling_latency_s":0.722333} vm-test-run-nixbot-gitlab> gitlab # [ 158.741931] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"284cf16a0b7a50f607689b5e","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-284cf16a0b7a50f607689b5e: done: 0.015481 sec","job_status":"done","queue_duration_s":0.722313,"scheduling_latency_s":0.722333,"redis_calls":13,"redis_duration_s":0.002771,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015481,"completed_at":"2026-06-29T22:07:33.652Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.782068] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae84be2f0e0ffe46fbb68fee","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae84be2f0e0ffe46fbb68fee: start","job_status":"start","queue_duration_s":0.739109,"scheduling_latency_s":0.73913} vm-test-run-nixbot-gitlab> gitlab # [ 158.854546] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae84be2f0e0ffe46fbb68fee","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae84be2f0e0ffe46fbb68fee: done: 0.015642 sec","job_status":"done","queue_duration_s":0.739109,"scheduling_latency_s":0.73913,"redis_calls":13,"redis_duration_s":0.002768,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015642,"completed_at":"2026-06-29T22:07:33.669Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.863162] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac9240760474e96a5db5b1ad","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac9240760474e96a5db5b1ad: start","job_status":"start","queue_duration_s":0.756118,"scheduling_latency_s":0.756141} vm-test-run-nixbot-gitlab> gitlab # [ 158.883279] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac9240760474e96a5db5b1ad","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac9240760474e96a5db5b1ad: done: 0.01793 sec","job_status":"done","queue_duration_s":0.756118,"scheduling_latency_s":0.756141,"redis_calls":13,"redis_duration_s":0.005326,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003338,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01793,"completed_at":"2026-06-29T22:07:33.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 158.993512] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81a8d9b46579e38538c3552c","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81a8d9b46579e38538c3552c: start","job_status":"start","queue_duration_s":0.776392,"scheduling_latency_s":0.776415} vm-test-run-nixbot-gitlab> gitlab # [ 158.995292] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81a8d9b46579e38538c3552c","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81a8d9b46579e38538c3552c: done: 0.022818 sec","job_status":"done","queue_duration_s":0.776392,"scheduling_latency_s":0.776415,"redis_calls":13,"redis_duration_s":0.009786,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002719,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022818,"completed_at":"2026-06-29T22:07:33.713Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.027227] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d655f415d4248c8a5a574a2d","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d655f415d4248c8a5a574a2d: start","job_status":"start","queue_duration_s":0.802319,"scheduling_latency_s":0.80234} vm-test-run-nixbot-gitlab> gitlab # [ 159.047285] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d655f415d4248c8a5a574a2d","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d655f415d4248c8a5a574a2d: done: 0.023172 sec","job_status":"done","queue_duration_s":0.802319,"scheduling_latency_s":0.80234,"redis_calls":13,"redis_duration_s":0.010049,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023172,"completed_at":"2026-06-29T22:07:33.739Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.148335] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9356d7d244812e078c361ad","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9356d7d244812e078c361ad: start","job_status":"start","queue_duration_s":0.827002,"scheduling_latency_s":0.827025} vm-test-run-nixbot-gitlab> gitlab # [ 159.149926] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9356d7d244812e078c361ad","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9356d7d244812e078c361ad: done: 0.015483 sec","job_status":"done","queue_duration_s":0.827002,"scheduling_latency_s":0.827025,"redis_calls":13,"redis_duration_s":0.002912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00191,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015483,"completed_at":"2026-06-29T22:07:33.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.232155] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6062468d610bcbe41a7fc8bc","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6062468d610bcbe41a7fc8bc: start","job_status":"start","queue_duration_s":0.843807,"scheduling_latency_s":0.843829} vm-test-run-nixbot-gitlab> gitlab # [ 159.233324] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6062468d610bcbe41a7fc8bc","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6062468d610bcbe41a7fc8bc: done: 0.01574 sec","job_status":"done","queue_duration_s":0.843807,"scheduling_latency_s":0.843829,"redis_calls":13,"redis_duration_s":0.00283,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01574,"completed_at":"2026-06-29T22:07:33.774Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.306252] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fda6e72de7e092130d1bd40","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fda6e72de7e092130d1bd40: start","job_status":"start","queue_duration_s":0.860993,"scheduling_latency_s":0.861014} vm-test-run-nixbot-gitlab> gitlab # [ 159.307486] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fda6e72de7e092130d1bd40","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fda6e72de7e092130d1bd40: done: 0.015412 sec","job_status":"done","queue_duration_s":0.860993,"scheduling_latency_s":0.861014,"redis_calls":13,"redis_duration_s":0.002757,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001885,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015412,"completed_at":"2026-06-29T22:07:33.790Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.353931] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54f903e2d97d9f3ea7df0e2b","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54f903e2d97d9f3ea7df0e2b: start","job_status":"start","queue_duration_s":0.877765,"scheduling_latency_s":0.877787} vm-test-run-nixbot-gitlab> gitlab # [ 159.423161] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54f903e2d97d9f3ea7df0e2b","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54f903e2d97d9f3ea7df0e2b: done: 0.017116 sec","job_status":"done","queue_duration_s":0.877765,"scheduling_latency_s":0.877787,"redis_calls":13,"redis_duration_s":0.003984,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017116,"completed_at":"2026-06-29T22:07:33.809Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.497339] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77c54a22377322f3a2b65865","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77c54a22377322f3a2b65865: start","job_status":"start","queue_duration_s":0.896124,"scheduling_latency_s":0.896145} vm-test-run-nixbot-gitlab> gitlab # [ 159.498960] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77c54a22377322f3a2b65865","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77c54a22377322f3a2b65865: done: 0.016053 sec","job_status":"done","queue_duration_s":0.896124,"scheduling_latency_s":0.896145,"redis_calls":13,"redis_duration_s":0.003229,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002221,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016053,"completed_at":"2026-06-29T22:07:33.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.571242] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad1c32e3614577bbfd394d46","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad1c32e3614577bbfd394d46: start","job_status":"start","queue_duration_s":0.913492,"scheduling_latency_s":0.913513} vm-test-run-nixbot-gitlab> gitlab # [ 159.572481] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad1c32e3614577bbfd394d46","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad1c32e3614577bbfd394d46: done: 0.016068 sec","job_status":"done","queue_duration_s":0.913492,"scheduling_latency_s":0.913513,"redis_calls":13,"redis_duration_s":0.003074,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002205,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016068,"completed_at":"2026-06-29T22:07:33.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.657916] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"539601339f443ea83e722974","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-539601339f443ea83e722974: start","job_status":"start","queue_duration_s":0.930914,"scheduling_latency_s":0.93098} vm-test-run-nixbot-gitlab> gitlab # [ 159.659338] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"539601339f443ea83e722974","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-539601339f443ea83e722974: done: 0.015884 sec","job_status":"done","queue_duration_s":0.930914,"scheduling_latency_s":0.93098,"redis_calls":13,"redis_duration_s":0.002982,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015884,"completed_at":"2026-06-29T22:07:33.861Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.734446] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ea291480aac82a5658e0582","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ea291480aac82a5658e0582: start","job_status":"start","queue_duration_s":0.948147,"scheduling_latency_s":0.948169} vm-test-run-nixbot-gitlab> gitlab # [ 159.736096] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ea291480aac82a5658e0582","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ea291480aac82a5658e0582: done: 0.016034 sec","job_status":"done","queue_duration_s":0.948147,"scheduling_latency_s":0.948169,"redis_calls":13,"redis_duration_s":0.003164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016034,"completed_at":"2026-06-29T22:07:33.878Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.760882] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47f10207e3934f783955a23a","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47f10207e3934f783955a23a: start","job_status":"start","queue_duration_s":0.965721,"scheduling_latency_s":0.965742} vm-test-run-nixbot-gitlab> gitlab # [ 159.780982] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47f10207e3934f783955a23a","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47f10207e3934f783955a23a: done: 0.01561 sec","job_status":"done","queue_duration_s":0.965721,"scheduling_latency_s":0.965742,"redis_calls":13,"redis_duration_s":0.0029000000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01561,"completed_at":"2026-06-29T22:07:33.895Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.855370] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:33.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd24ca0dfc0c2b5710f9a07a","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd24ca0dfc0c2b5710f9a07a: start","job_status":"start","queue_duration_s":0.982727,"scheduling_latency_s":0.982748} vm-test-run-nixbot-gitlab> gitlab # [ 159.862279] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd24ca0dfc0c2b5710f9a07a","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd24ca0dfc0c2b5710f9a07a: done: 0.279608 sec","job_status":"done","queue_duration_s":0.982727,"scheduling_latency_s":0.982748,"redis_calls":13,"redis_duration_s":0.003173,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00116,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.279608,"completed_at":"2026-06-29T22:07:34.176Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.985360] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.179Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcda3c85f14e27c6e6cd5975","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcda3c85f14e27c6e6cd5975: start","job_status":"start","queue_duration_s":1.264525,"scheduling_latency_s":1.264556} vm-test-run-nixbot-gitlab> gitlab # [ 159.986510] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcda3c85f14e27c6e6cd5975","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcda3c85f14e27c6e6cd5975: done: 0.018923 sec","job_status":"done","queue_duration_s":1.264525,"scheduling_latency_s":1.264556,"redis_calls":13,"redis_duration_s":0.005433,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001738,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003695,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018923,"completed_at":"2026-06-29T22:07:34.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.006544] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c657378b4e07f91a08454e8c","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c657378b4e07f91a08454e8c: start","job_status":"start","queue_duration_s":1.285436,"scheduling_latency_s":1.285458} vm-test-run-nixbot-gitlab> gitlab # [ 160.026241] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c657378b4e07f91a08454e8c","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c657378b4e07f91a08454e8c: done: 0.018489 sec","job_status":"done","queue_duration_s":1.285436,"scheduling_latency_s":1.285458,"redis_calls":13,"redis_duration_s":0.005943,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002121,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003822,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018489,"completed_at":"2026-06-29T22:07:34.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.148404] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be751e6ceec953da003f0063","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be751e6ceec953da003f0063: start","job_status":"start","queue_duration_s":1.305885,"scheduling_latency_s":1.305906} vm-test-run-nixbot-gitlab> gitlab # [ 160.149875] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be751e6ceec953da003f0063","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be751e6ceec953da003f0063: done: 0.017601 sec","job_status":"done","queue_duration_s":1.305885,"scheduling_latency_s":1.305906,"redis_calls":13,"redis_duration_s":0.005238,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001767,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003471,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017601,"completed_at":"2026-06-29T22:07:34.237Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.170339] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15158a518bfe1c687d27708c","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15158a518bfe1c687d27708c: start","job_status":"start","queue_duration_s":1.325484,"scheduling_latency_s":1.325506} vm-test-run-nixbot-gitlab> gitlab # [ 160.190387] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15158a518bfe1c687d27708c","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15158a518bfe1c687d27708c: done: 0.034055 sec","job_status":"done","queue_duration_s":1.325484,"scheduling_latency_s":1.325506,"redis_calls":13,"redis_duration_s":0.005941999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001611,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004331,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034055,"completed_at":"2026-06-29T22:07:34.274Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.260101] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f312c328443c99daac404416","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f312c328443c99daac404416: start","job_status":"start","queue_duration_s":1.361209,"scheduling_latency_s":1.361235} vm-test-run-nixbot-gitlab> gitlab # [ 160.333282] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:34.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f312c328443c99daac404416","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f312c328443c99daac404416: done: 0.01691 sec","job_status":"done","queue_duration_s":1.361209,"scheduling_latency_s":1.361235,"redis_calls":13,"redis_duration_s":0.002875,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01691,"completed_at":"2026-06-29T22:07:34.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.551710] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0814b65b679db46cbcfa6f80","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0814b65b679db46cbcfa6f80: start","job_status":"start","queue_duration_s":4.542306,"scheduling_latency_s":4.542381} vm-test-run-nixbot-gitlab> gitlab # [ 160.640249] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0814b65b679db46cbcfa6f80","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0814b65b679db46cbcfa6f80: done: 0.090257 sec","job_status":"done","queue_duration_s":4.542306,"scheduling_latency_s":4.542381,"redis_calls":13,"redis_duration_s":0.008566,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002367,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.090257,"completed_at":"2026-06-29T22:07:37.546Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.726285] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77b245c7e099f6ab506c706f","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77b245c7e099f6ab506c706f: start","job_status":"start","queue_duration_s":4.718462,"scheduling_latency_s":4.718503} vm-test-run-nixbot-gitlab> gitlab # [ 160.791213] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77b245c7e099f6ab506c706f","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77b245c7e099f6ab506c706f: done: 0.064774 sec","job_status":"done","queue_duration_s":4.718462,"scheduling_latency_s":4.718503,"redis_calls":13,"redis_duration_s":0.008365000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005417,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.064774,"completed_at":"2026-06-29T22:07:37.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.861349] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec7bf0773b1f5b5f13249488","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec7bf0773b1f5b5f13249488: start","job_status":"start","queue_duration_s":4.853335,"scheduling_latency_s":4.853411} vm-test-run-nixbot-gitlab> gitlab # [ 160.918205] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec7bf0773b1f5b5f13249488","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec7bf0773b1f5b5f13249488: done: 0.057039 sec","job_status":"done","queue_duration_s":4.853335,"scheduling_latency_s":4.853411,"redis_calls":13,"redis_duration_s":0.0051459999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003446,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.057039,"completed_at":"2026-06-29T22:07:37.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.984151] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"431db071ac80d085b044c182","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-431db071ac80d085b044c182: start","job_status":"start","queue_duration_s":4.976296,"scheduling_latency_s":4.976366} vm-test-run-nixbot-gitlab> gitlab # [ 161.027377] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"431db071ac80d085b044c182","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-431db071ac80d085b044c182: done: 0.043004 sec","job_status":"done","queue_duration_s":4.976296,"scheduling_latency_s":4.976366,"redis_calls":13,"redis_duration_s":0.008333,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005796,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043004,"completed_at":"2026-06-29T22:07:37.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.090770] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74e7dfa2491fe7c66fa9882c","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74e7dfa2491fe7c66fa9882c: start","job_status":"start","queue_duration_s":5.022666,"scheduling_latency_s":5.022717} vm-test-run-nixbot-gitlab> gitlab # [ 161.091336] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74e7dfa2491fe7c66fa9882c","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74e7dfa2491fe7c66fa9882c: done: 0.031489 sec","job_status":"done","queue_duration_s":5.022666,"scheduling_latency_s":5.022717,"redis_calls":13,"redis_duration_s":0.006834999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004768,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031489,"completed_at":"2026-06-29T22:07:37.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.102927] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a9cb4f766a0b6227c3726d1","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a9cb4f766a0b6227c3726d1: start","job_status":"start","queue_duration_s":5.056032,"scheduling_latency_s":5.056055} vm-test-run-nixbot-gitlab> gitlab # [ 161.124187] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a9cb4f766a0b6227c3726d1","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a9cb4f766a0b6227c3726d1: done: 0.018343 sec","job_status":"done","queue_duration_s":5.056032,"scheduling_latency_s":5.056055,"redis_calls":13,"redis_duration_s":0.005417,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001779,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003638,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018343,"completed_at":"2026-06-29T22:07:37.988Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.187347] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:37.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0a67972a882d7faf9b2cb19","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0a67972a882d7faf9b2cb19: start","job_status":"start","queue_duration_s":5.07689,"scheduling_latency_s":5.076911} vm-test-run-nixbot-gitlab> gitlab # [ 161.303440] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0a67972a882d7faf9b2cb19","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0a67972a882d7faf9b2cb19: done: 0.218394 sec","job_status":"done","queue_duration_s":5.07689,"scheduling_latency_s":5.076911,"redis_calls":13,"redis_duration_s":0.187794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.1846,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.218394,"completed_at":"2026-06-29T22:07:38.209Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.369323] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828546045a9a067bf2c39aa1","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828546045a9a067bf2c39aa1: start","job_status":"start","queue_duration_s":5.361293,"scheduling_latency_s":5.36137} vm-test-run-nixbot-gitlab> gitlab # [ 161.413794] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828546045a9a067bf2c39aa1","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828546045a9a067bf2c39aa1: done: 0.043445 sec","job_status":"done","queue_duration_s":5.361293,"scheduling_latency_s":5.36137,"redis_calls":13,"redis_duration_s":0.006562999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004578,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043445,"completed_at":"2026-06-29T22:07:38.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.479310] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f211c2fa53fc8c3af423db63","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f211c2fa53fc8c3af423db63: start","job_status":"start","queue_duration_s":5.471322,"scheduling_latency_s":5.471395} vm-test-run-nixbot-gitlab> gitlab # [ 161.539873] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f211c2fa53fc8c3af423db63","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f211c2fa53fc8c3af423db63: done: 0.060773 sec","job_status":"done","queue_duration_s":5.471322,"scheduling_latency_s":5.471395,"redis_calls":13,"redis_duration_s":0.006992,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001687,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005305,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.060773,"completed_at":"2026-06-29T22:07:38.446Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.606490] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604a93c3b82a4eb553f74eec","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604a93c3b82a4eb553f74eec: start","job_status":"start","queue_duration_s":5.598525,"scheduling_latency_s":5.59859} vm-test-run-nixbot-gitlab> gitlab # [ 161.662981] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604a93c3b82a4eb553f74eec","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604a93c3b82a4eb553f74eec: done: 0.056486 sec","job_status":"done","queue_duration_s":5.598525,"scheduling_latency_s":5.59859,"redis_calls":13,"redis_duration_s":0.005065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001706,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003359,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.056486,"completed_at":"2026-06-29T22:07:38.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.729358] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bca3459dbcd17ca08b024501","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bca3459dbcd17ca08b024501: start","job_status":"start","queue_duration_s":5.721336,"scheduling_latency_s":5.721407} vm-test-run-nixbot-gitlab> gitlab # [ 161.791309] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bca3459dbcd17ca08b024501","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bca3459dbcd17ca08b024501: done: 0.062005 sec","job_status":"done","queue_duration_s":5.721336,"scheduling_latency_s":5.721407,"redis_calls":13,"redis_duration_s":0.013277999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005449,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007829,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.062005,"completed_at":"2026-06-29T22:07:38.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.858765] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e691c58ce640aad3c95960d","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e691c58ce640aad3c95960d: start","job_status":"start","queue_duration_s":5.849637,"scheduling_latency_s":5.849712} vm-test-run-nixbot-gitlab> gitlab # [ 161.902991] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e691c58ce640aad3c95960d","created_at":"2026-06-29T22:07:32.842Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-578780eeb29d4da2","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3efb21aedb113122292ada4411f4b438","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:32.914Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e691c58ce640aad3c95960d: done: 0.045618 sec","job_status":"done","queue_duration_s":5.849637,"scheduling_latency_s":5.849712,"redis_calls":13,"redis_duration_s":0.008126,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00541,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.045618,"completed_at":"2026-06-29T22:07:38.809Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.969880] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.876Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0ab542942884ad4d06196bce","created_at":"2026-06-29T22:07:33.165Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-7b32bdf4a75340f6","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:33.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0ab542942884ad4d06196bce: start","job_status":"start","queue_duration_s":5.700814,"scheduling_latency_s":5.700984} vm-test-run-nixbot-gitlab> gitlab # [ 162.007423] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.914Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 162.031973] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:38.937Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0ab542942884ad4d06196bce","created_at":"2026-06-29T22:07:33.165Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-7b32bdf4a75340f6","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:33.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0ab542942884ad4d06196bce: done: 0.061512 sec","job_status":"done","queue_duration_s":5.700814,"scheduling_latency_s":5.700984,"redis_calls":13,"redis_duration_s":0.008466,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000955149,"duration_s":0.061512,"completed_at":"2026-06-29T22:07:38.937Z","load_balancing_strategy":"primary","db_duration_s":0.008037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.097527] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:39.004Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"234135a91d6199e98ae8b4bd","created_at":"2026-06-29T22:07:33.165Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-7b32bdf4a75340f6","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:33.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-234135a91d6199e98ae8b4bd: start","job_status":"start","queue_duration_s":5.828405,"scheduling_latency_s":5.828475} vm-test-run-nixbot-gitlab> gitlab # [ 162.140832] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:39.047Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 162.159473] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:39.065Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"234135a91d6199e98ae8b4bd","created_at":"2026-06-29T22:07:33.165Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-7b32bdf4a75340f6","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:33.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-234135a91d6199e98ae8b4bd: done: 0.062029 sec","job_status":"done","queue_duration_s":5.828405,"scheduling_latency_s":5.828475,"redis_calls":10,"redis_duration_s":0.006895,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00159,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005305,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.062029,"completed_at":"2026-06-29T22:07:39.065Z","load_balancing_strategy":"primary","db_duration_s":0.005081,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.225101] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:39.130Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5787f26e0829858bbc7aee02","created_at":"2026-06-29T22:07:33.165Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-7b32bdf4a75340f6","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:33.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5787f26e0829858bbc7aee02: start","job_status":"start","queue_duration_s":5.954799,"scheduling_latency_s":5.954962} vm-test-run-nixbot-gitlab> gitlab # [ 162.253231] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:39.160Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 162.272322] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:39.178Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5787f26e0829858bbc7aee02","created_at":"2026-06-29T22:07:33.165Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-7b32bdf4a75340f6","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:33.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5787f26e0829858bbc7aee02: done: 0.048207 sec","job_status":"done","queue_duration_s":5.954799,"scheduling_latency_s":5.954962,"redis_calls":10,"redis_duration_s":0.007828,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002146,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005682,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048207,"completed_at":"2026-06-29T22:07:39.178Z","load_balancing_strategy":"primary","db_duration_s":0.0051,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.336763] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:39.243Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e04f9de61a71dcf4372a5b61","created_at":"2026-06-29T22:07:33.165Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-7b32bdf4a75340f6","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:33.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e04f9de61a71dcf4372a5b61: start","job_status":"start","queue_duration_s":6.067673,"scheduling_latency_s":6.06774} vm-test-run-nixbot-gitlab> gitlab # [ 162.363929] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:39.271Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 162.384131] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:07:39.290Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e04f9de61a71dcf4372a5b61","created_at":"2026-06-29T22:07:33.165Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-7b32bdf4a75340f6","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"95d65a0089fc354aca84fff66781e4a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:07:33.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e04f9de61a71dcf4372a5b61: done: 0.047363 sec","job_status":"done","queue_duration_s":6.067673,"scheduling_latency_s":6.06774,"redis_calls":10,"redis_duration_s":0.009518,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00236,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.007158,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047363,"completed_at":"2026-06-29T22:07:39.290Z","load_balancing_strategy":"primary","db_duration_s":0.004925,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 174.390483] setup-gitlab-start[2736]: W, [2026-06-29T22:07:51.296635 #2736] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 188.108958] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.014Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6abd52bba3a15c0e9ae3bdb2","created_at":"2026-06-29T22:08:04.950Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a09149dc1c2b1331fafe8dae7de3dd20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.011Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6abd52bba3a15c0e9ae3bdb2: start","job_status":"start","queue_duration_s":0.003137,"scheduling_latency_s":0.00319} vm-test-run-nixbot-gitlab> gitlab # [ 188.185954] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6abd52bba3a15c0e9ae3bdb2","created_at":"2026-06-29T22:08:04.950Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a09149dc1c2b1331fafe8dae7de3dd20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.011Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6abd52bba3a15c0e9ae3bdb2: done: 0.078091 sec","job_status":"done","queue_duration_s":0.003137,"scheduling_latency_s":0.00319,"redis_calls":8,"redis_duration_s":0.015167,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000702,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007346,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007119,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.078091,"completed_at":"2026-06-29T22:08:05.092Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 188.238591] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9b213da3307bb4bfcf42e177","created_at":"2026-06-29T22:08:05.036Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cefe7fef6c09d1aea161cedbae7e9f27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.050Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9b213da3307bb4bfcf42e177: start","job_status":"start","queue_duration_s":0.065794,"scheduling_latency_s":0.065874} vm-test-run-nixbot-gitlab> gitlab # [ 188.615219] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9b213da3307bb4bfcf42e177","created_at":"2026-06-29T22:08:05.036Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cefe7fef6c09d1aea161cedbae7e9f27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.050Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9b213da3307bb4bfcf42e177: done: 0.405381 sec","job_status":"done","queue_duration_s":0.065794,"scheduling_latency_s":0.065874,"redis_calls":11,"redis_duration_s":0.006983,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001723,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002555,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.072,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.072,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.14779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000934755,"exclusive_lock_hold_duration_s":0.264438637,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.405381,"completed_at":"2026-06-29T22:08:05.521Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.178482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 188.714995] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1c146d83fc0115d099b99174","created_at":"2026-06-29T22:08:05.078Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"089105ea7236a280449f5aa4366ecd68","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.087Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1c146d83fc0115d099b99174: start","job_status":"start","queue_duration_s":0.436396,"scheduling_latency_s":0.436419} vm-test-run-nixbot-gitlab> gitlab # [ 188.737856] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.533Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 188.743556] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1c146d83fc0115d099b99174","created_at":"2026-06-29T22:08:05.078Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"089105ea7236a280449f5aa4366ecd68","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.087Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1c146d83fc0115d099b99174: done: 0.014793 sec","job_status":"done","queue_duration_s":0.436396,"scheduling_latency_s":0.436419,"redis_calls":7,"redis_duration_s":0.003101,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014793,"completed_at":"2026-06-29T22:08:05.538Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 188.745097] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.540Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"9b96e049f2037d532c3720f9","created_at":"2026-06-29T22:08:05.117Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"821962169bbb56ae17d7c7779069f9e2","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.150Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-9b96e049f2037d532c3720f9: start","job_status":"start","queue_duration_s":0.390018,"scheduling_latency_s":0.390042} vm-test-run-nixbot-gitlab> gitlab # [ 188.760050] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.571Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"9b96e049f2037d532c3720f9","created_at":"2026-06-29T22:08:05.117Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"821962169bbb56ae17d7c7779069f9e2","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.150Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-9b96e049f2037d532c3720f9: done: 0.030471 sec","job_status":"done","queue_duration_s":0.390018,"scheduling_latency_s":0.390042,"redis_calls":10,"redis_duration_s":0.005117,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001476,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002758,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000883,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000667124,"duration_s":0.030471,"completed_at":"2026-06-29T22:08:05.571Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 188.827602] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.573Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"df61e4ec0d5eb1ea1d641d94","created_at":"2026-06-29T22:08:05.156Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13647834854ea51084429dde1a3235c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T22:08:05.157Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-df61e4ec0d5eb1ea1d641d94: start","job_status":"start","queue_duration_s":0.415564,"scheduling_latency_s":0.415587} vm-test-run-nixbot-gitlab> gitlab # [ 188.848805] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.582Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 188.854094] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.587Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"df61e4ec0d5eb1ea1d641d94","created_at":"2026-06-29T22:08:05.156Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13647834854ea51084429dde1a3235c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T22:08:05.157Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-df61e4ec0d5eb1ea1d641d94: done: 0.014331 sec","job_status":"done","queue_duration_s":0.415564,"scheduling_latency_s":0.415587,"redis_calls":7,"redis_duration_s":0.003327,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014331,"completed_at":"2026-06-29T22:08:05.587Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 188.958098] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8187bb17240634287b74ac4d","created_at":"2026-06-29T22:08:05.162Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52baac554574644eef9bc72df4996957","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.163Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8187bb17240634287b74ac4d: start","job_status":"start","queue_duration_s":0.425861,"scheduling_latency_s":0.425883} vm-test-run-nixbot-gitlab> gitlab # [ 188.959438] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.631Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8187bb17240634287b74ac4d","created_at":"2026-06-29T22:08:05.162Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52baac554574644eef9bc72df4996957","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.163Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8187bb17240634287b74ac4d: done: 0.04169 sec","job_status":"done","queue_duration_s":0.425861,"scheduling_latency_s":0.425883,"redis_calls":9,"redis_duration_s":0.004288,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002127,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000825524,"exclusive_lock_hold_duration_s":0.015361729,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.04169,"completed_at":"2026-06-29T22:08:05.631Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019385,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.036439] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a6d30ec84b510982b0ca2748","created_at":"2026-06-29T22:08:05.173Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5367061255c5b9e0f8a7a43f57bca401","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a6d30ec84b510982b0ca2748: start","job_status":"start","queue_duration_s":0.458396,"scheduling_latency_s":0.458453} vm-test-run-nixbot-gitlab> gitlab # [ 189.037899] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a6d30ec84b510982b0ca2748","created_at":"2026-06-29T22:08:05.173Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5367061255c5b9e0f8a7a43f57bca401","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.175Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a6d30ec84b510982b0ca2748: done: 0.032723 sec","job_status":"done","queue_duration_s":0.458396,"scheduling_latency_s":0.458453,"redis_calls":7,"redis_duration_s":0.003576,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002569,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.032723,"completed_at":"2026-06-29T22:08:05.666Z","load_balancing_strategy":"primary","db_duration_s":0.010912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.121772] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ed53d4593f092f4480d2dac7","created_at":"2026-06-29T22:08:05.178Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfd4f360f44f9588bfe2759c72d10ba3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.180Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ed53d4593f092f4480d2dac7: start","job_status":"start","queue_duration_s":0.488994,"scheduling_latency_s":0.489043} vm-test-run-nixbot-gitlab> gitlab # [ 189.123145] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.677Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 189.123917] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ed53d4593f092f4480d2dac7","created_at":"2026-06-29T22:08:05.178Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfd4f360f44f9588bfe2759c72d10ba3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.180Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ed53d4593f092f4480d2dac7: done: 0.013002 sec","job_status":"done","queue_duration_s":0.488994,"scheduling_latency_s":0.489043,"redis_calls":7,"redis_duration_s":0.001667,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001178,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013002,"completed_at":"2026-06-29T22:08:05.681Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.200293] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.683Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"6e2b12a15bad7ca40959ebcc","created_at":"2026-06-29T22:08:05.185Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"6cb7103494e9470f6707415da963152f","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.191Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-6e2b12a15bad7ca40959ebcc: start","job_status":"start","queue_duration_s":0.49178,"scheduling_latency_s":0.491801} vm-test-run-nixbot-gitlab> gitlab # [ 189.204914] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.700Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"6e2b12a15bad7ca40959ebcc","created_at":"2026-06-29T22:08:05.185Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"6cb7103494e9470f6707415da963152f","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.191Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-6e2b12a15bad7ca40959ebcc: done: 0.016776 sec","job_status":"done","queue_duration_s":0.49178,"scheduling_latency_s":0.491801,"redis_calls":10,"redis_duration_s":0.002365,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001423,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000427708,"duration_s":0.016776,"completed_at":"2026-06-29T22:08:05.700Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.278153] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:05.701Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0a8338e50832dc3d49c2109a","created_at":"2026-06-29T22:08:05.197Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"437635d19d1c2c81892766521d2ded03","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.200Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0a8338e50832dc3d49c2109a: start","job_status":"start","queue_duration_s":0.500617,"scheduling_latency_s":0.50068} vm-test-run-nixbot-gitlab> gitlab # [ 189.291864] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.113Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0a8338e50832dc3d49c2109a","created_at":"2026-06-29T22:08:05.197Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"437635d19d1c2c81892766521d2ded03","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.200Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0a8338e50832dc3d49c2109a: done: 0.412322 sec","job_status":"done","queue_duration_s":0.500617,"scheduling_latency_s":0.50068,"redis_calls":571,"redis_duration_s":0.138327,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.133249,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.313171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.412322,"completed_at":"2026-06-29T22:08:06.113Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.381226] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.115Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"5c1cdfc01ce27e5b3c7a802a","created_at":"2026-06-29T22:08:05.218Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aab536e6292436623bd2d590ead87821","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-5c1cdfc01ce27e5b3c7a802a: start","job_status":"start","queue_duration_s":0.891677,"scheduling_latency_s":0.891707} vm-test-run-nixbot-gitlab> gitlab # [ 189.382710] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.134Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"5c1cdfc01ce27e5b3c7a802a","created_at":"2026-06-29T22:08:05.218Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aab536e6292436623bd2d590ead87821","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-5c1cdfc01ce27e5b3c7a802a: done: 0.019017 sec","job_status":"done","queue_duration_s":0.891677,"scheduling_latency_s":0.891707,"redis_calls":12,"redis_duration_s":0.0030139999999999998,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000535,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000466,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000563,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000361777,"duration_s":0.019017,"completed_at":"2026-06-29T22:08:06.134Z","load_balancing_strategy":"primary","db_duration_s":0.000684,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.466064] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d5d7d876a80262fa3a569ff9","created_at":"2026-06-29T22:08:05.232Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.234Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d5d7d876a80262fa3a569ff9: start","job_status":"start","queue_duration_s":0.901778,"scheduling_latency_s":0.9018} vm-test-run-nixbot-gitlab> gitlab # [ 189.467548] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d5d7d876a80262fa3a569ff9","created_at":"2026-06-29T22:08:05.232Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.234Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d5d7d876a80262fa3a569ff9: done: 0.094518 sec","job_status":"done","queue_duration_s":0.901778,"scheduling_latency_s":0.9018,"redis_calls":60,"redis_duration_s":0.030917,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000887,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.029367,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000663,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.094518,"completed_at":"2026-06-29T22:08:06.230Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.589654] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.231Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7421137ebb47bebaadfbcd89","created_at":"2026-06-29T22:08:05.242Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f5928ad4ce135d9ee9f6909d8cfac45","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.244Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7421137ebb47bebaadfbcd89: start","job_status":"start","queue_duration_s":0.987499,"scheduling_latency_s":0.987528} vm-test-run-nixbot-gitlab> gitlab # [ 189.591281] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7421137ebb47bebaadfbcd89","created_at":"2026-06-29T22:08:05.242Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f5928ad4ce135d9ee9f6909d8cfac45","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.244Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7421137ebb47bebaadfbcd89: done: 0.013937 sec","job_status":"done","queue_duration_s":0.987499,"scheduling_latency_s":0.987528,"redis_calls":7,"redis_duration_s":0.001582,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013937,"completed_at":"2026-06-29T22:08:06.245Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001278,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.608443] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.247Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"6a78064d54326c3faa2dcaf0","created_at":"2026-06-29T22:08:05.253Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06615135531b92a93e52c02eeef56568","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T22:08:05.256Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-6a78064d54326c3faa2dcaf0: start","job_status":"start","queue_duration_s":0.99011,"scheduling_latency_s":0.990132} vm-test-run-nixbot-gitlab> gitlab # [ 189.630105] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.254Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 189.636067] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"6a78064d54326c3faa2dcaf0","created_at":"2026-06-29T22:08:05.253Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06615135531b92a93e52c02eeef56568","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T22:08:05.256Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-6a78064d54326c3faa2dcaf0: done: 0.011662 sec","job_status":"done","queue_duration_s":0.99011,"scheduling_latency_s":0.990132,"redis_calls":7,"redis_duration_s":0.0015660000000000001,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011662,"completed_at":"2026-06-29T22:08:06.258Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.762464] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.260Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c1150f73e98de4f3dab7e619","created_at":"2026-06-29T22:08:05.260Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ea2d0862268decd39e6b17836d32b07","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T22:08:05.262Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c1150f73e98de4f3dab7e619: start","job_status":"start","queue_duration_s":0.997701,"scheduling_latency_s":0.997722} vm-test-run-nixbot-gitlab> gitlab # [ 189.763974] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.277Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c1150f73e98de4f3dab7e619","created_at":"2026-06-29T22:08:05.260Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ea2d0862268decd39e6b17836d32b07","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T22:08:05.262Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c1150f73e98de4f3dab7e619: done: 0.017519 sec","job_status":"done","queue_duration_s":0.997701,"scheduling_latency_s":0.997722,"redis_calls":7,"redis_duration_s":0.001697,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017519,"completed_at":"2026-06-29T22:08:06.277Z","load_balancing_strategy":"primary","db_duration_s":0.001381,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.780128] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1ac131fda8e41f0ccfbbaaa9","created_at":"2026-06-29T22:08:05.265Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81e1f3e8cb35e2f2112112ede1cfd315","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.268Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1ac131fda8e41f0ccfbbaaa9: start","job_status":"start","queue_duration_s":1.010086,"scheduling_latency_s":1.010107} vm-test-run-nixbot-gitlab> gitlab # [ 189.803366] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.293Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1ac131fda8e41f0ccfbbaaa9","created_at":"2026-06-29T22:08:05.265Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81e1f3e8cb35e2f2112112ede1cfd315","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.268Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1ac131fda8e41f0ccfbbaaa9: done: 0.014679 sec","job_status":"done","queue_duration_s":1.010086,"scheduling_latency_s":1.010107,"redis_calls":7,"redis_duration_s":0.0042049999999999995,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000602,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014679,"completed_at":"2026-06-29T22:08:06.293Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.874136] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.296Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"4d2d9efc5cf1b50c39065121","created_at":"2026-06-29T22:08:05.275Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f25099f51afb1c97363f34df8896c4c3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.284Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-4d2d9efc5cf1b50c39065121: start","job_status":"start","queue_duration_s":1.012193,"scheduling_latency_s":1.012216} vm-test-run-nixbot-gitlab> gitlab # [ 189.942584] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"4d2d9efc5cf1b50c39065121","created_at":"2026-06-29T22:08:05.275Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f25099f51afb1c97363f34df8896c4c3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.284Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-4d2d9efc5cf1b50c39065121: done: 0.02211 sec","job_status":"done","queue_duration_s":1.012193,"scheduling_latency_s":1.012216,"redis_calls":12,"redis_duration_s":0.0049440000000000005,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001189,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000465422,"exclusive_lock_hold_duration_s":0.0037015869999999997,"duration_s":0.02211,"completed_at":"2026-06-29T22:08:06.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.956737] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.320Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1cb2ae59ff33ae8ff65bae9c","created_at":"2026-06-29T22:08:05.300Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8aeee6c6446bf5e2940d7367c52cd1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.303Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1cb2ae59ff33ae8ff65bae9c: start","job_status":"start","queue_duration_s":1.016816,"scheduling_latency_s":1.016836} vm-test-run-nixbot-gitlab> gitlab # [ 190.042665] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1cb2ae59ff33ae8ff65bae9c","created_at":"2026-06-29T22:08:05.300Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8aeee6c6446bf5e2940d7367c52cd1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.303Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1cb2ae59ff33ae8ff65bae9c: done: 0.013795 sec","job_status":"done","queue_duration_s":1.016816,"scheduling_latency_s":1.016836,"redis_calls":9,"redis_duration_s":0.002086,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000490006,"exclusive_lock_hold_duration_s":0.001870629,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.013795,"completed_at":"2026-06-29T22:08:06.333Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.045047] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.335Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2fb38e524598c00e82392c54","created_at":"2026-06-29T22:08:05.312Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c98d3f24a4595b38c7ffb952f130cc68","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.313Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2fb38e524598c00e82392c54: start","job_status":"start","queue_duration_s":1.021337,"scheduling_latency_s":1.021357} vm-test-run-nixbot-gitlab> gitlab # [ 190.116651] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.352Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2fb38e524598c00e82392c54","created_at":"2026-06-29T22:08:05.312Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c98d3f24a4595b38c7ffb952f130cc68","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.313Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2fb38e524598c00e82392c54: done: 0.016997 sec","job_status":"done","queue_duration_s":1.021337,"scheduling_latency_s":1.021357,"redis_calls":7,"redis_duration_s":0.001633,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001183,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00045,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016997,"completed_at":"2026-06-29T22:08:06.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003311,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.131721] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.353Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"39509eb24646176929a9a0e7","created_at":"2026-06-29T22:08:05.339Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce2ff5fdc23680d72cb2f55fdb1cd633","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.343Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-39509eb24646176929a9a0e7: start","job_status":"start","queue_duration_s":1.00976,"scheduling_latency_s":1.009789} vm-test-run-nixbot-gitlab> gitlab # [ 190.208170] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"39509eb24646176929a9a0e7","created_at":"2026-06-29T22:08:05.339Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce2ff5fdc23680d72cb2f55fdb1cd633","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.343Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-39509eb24646176929a9a0e7: done: 0.014141 sec","job_status":"done","queue_duration_s":1.00976,"scheduling_latency_s":1.009789,"redis_calls":9,"redis_duration_s":0.002075,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001108,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000445028,"exclusive_lock_hold_duration_s":0.002264534,"duration_s":0.014141,"completed_at":"2026-06-29T22:08:06.367Z","load_balancing_strategy":"primary","db_duration_s":0.000868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.215319] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.369Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"03146800e627d604f2994d9b","created_at":"2026-06-29T22:08:05.353Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1d33871d2fc21560377af600c5d9f20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.355Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-03146800e627d604f2994d9b: start","job_status":"start","queue_duration_s":1.013354,"scheduling_latency_s":1.013374} vm-test-run-nixbot-gitlab> gitlab # [ 190.296529] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.378Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 190.298195] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"03146800e627d604f2994d9b","created_at":"2026-06-29T22:08:05.353Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1d33871d2fc21560377af600c5d9f20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.355Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-03146800e627d604f2994d9b: done: 0.015294 sec","job_status":"done","queue_duration_s":1.013354,"scheduling_latency_s":1.013374,"redis_calls":7,"redis_duration_s":0.00528,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002858,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002422,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015294,"completed_at":"2026-06-29T22:08:06.384Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.373378] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c2baa24c1e4b57583a353b7b","created_at":"2026-06-29T22:08:05.372Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.375Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c2baa24c1e4b57583a353b7b: start","job_status":"start","queue_duration_s":1.012028,"scheduling_latency_s":1.012051} vm-test-run-nixbot-gitlab> gitlab # [ 190.374955] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c2baa24c1e4b57583a353b7b","created_at":"2026-06-29T22:08:05.372Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.375Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c2baa24c1e4b57583a353b7b: done: 0.027862 sec","job_status":"done","queue_duration_s":1.012028,"scheduling_latency_s":1.012051,"redis_calls":14,"redis_duration_s":0.010102,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.00098,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.006974,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002148,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027862,"completed_at":"2026-06-29T22:08:06.415Z","load_balancing_strategy":"primary","db_duration_s":0.001247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.393331] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2fb452fe5b52fb3b8c4e2be6","created_at":"2026-06-29T22:08:05.391Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18a7ca60a22aaba6d21ced72ac516cb5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.393Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2fb452fe5b52fb3b8c4e2be6: start","job_status":"start","queue_duration_s":1.025803,"scheduling_latency_s":1.025825} vm-test-run-nixbot-gitlab> gitlab # [ 190.463190] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2fb452fe5b52fb3b8c4e2be6","created_at":"2026-06-29T22:08:05.391Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18a7ca60a22aaba6d21ced72ac516cb5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.393Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2fb452fe5b52fb3b8c4e2be6: done: 0.115 sec","job_status":"done","queue_duration_s":1.025803,"scheduling_latency_s":1.025825,"redis_calls":9,"redis_duration_s":0.002661,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001413,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051906,"exclusive_lock_hold_duration_s":0.100373371,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.115,"completed_at":"2026-06-29T22:08:06.533Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.094972,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.546091] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"58461ebb47f5977c34e7ee87","created_at":"2026-06-29T22:08:05.400Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a003a7c7a16197fba27da690c055924","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.401Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-58461ebb47f5977c34e7ee87: start","job_status":"start","queue_duration_s":1.133726,"scheduling_latency_s":1.133749} vm-test-run-nixbot-gitlab> gitlab # [ 190.547269] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.543Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 190.548235] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.547Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"58461ebb47f5977c34e7ee87","created_at":"2026-06-29T22:08:05.400Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a003a7c7a16197fba27da690c055924","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.401Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-58461ebb47f5977c34e7ee87: done: 0.012285 sec","job_status":"done","queue_duration_s":1.133726,"scheduling_latency_s":1.133749,"redis_calls":7,"redis_duration_s":0.001625,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012285,"completed_at":"2026-06-29T22:08:06.547Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.625044] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"5990f8b7c7e1b20a25cf01ba","created_at":"2026-06-29T22:08:05.407Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f39e023cd5fbc9b88cba8111fe2e100","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.412Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-5990f8b7c7e1b20a25cf01ba: start","job_status":"start","queue_duration_s":1.136257,"scheduling_latency_s":1.136279} vm-test-run-nixbot-gitlab> gitlab # [ 190.629078] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.568Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"5990f8b7c7e1b20a25cf01ba","created_at":"2026-06-29T22:08:05.407Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-6e5ae4480b6a453a","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f39e023cd5fbc9b88cba8111fe2e100","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:05.412Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-5990f8b7c7e1b20a25cf01ba: done: 0.019538 sec","job_status":"done","queue_duration_s":1.136257,"scheduling_latency_s":1.136279,"redis_calls":10,"redis_duration_s":0.0025310000000000003,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001484,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005843201,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.019538,"completed_at":"2026-06-29T22:08:06.568Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004517,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 190.706944] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6328a2966659706233669f0a","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6328a2966659706233669f0a: start","job_status":"start","queue_duration_s":0.346626,"scheduling_latency_s":0.346646} vm-test-run-nixbot-gitlab> gitlab # [ 190.708232] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6328a2966659706233669f0a","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6328a2966659706233669f0a: done: 0.015377 sec","job_status":"done","queue_duration_s":0.346626,"scheduling_latency_s":0.346646,"redis_calls":13,"redis_duration_s":0.00312,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015377,"completed_at":"2026-06-29T22:08:06.585Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.803156] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b2baabe08ed3ee73c733f41","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b2baabe08ed3ee73c733f41: start","job_status":"start","queue_duration_s":0.363396,"scheduling_latency_s":0.363418} vm-test-run-nixbot-gitlab> gitlab # [ 190.804488] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b2baabe08ed3ee73c733f41","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b2baabe08ed3ee73c733f41: done: 0.015161 sec","job_status":"done","queue_duration_s":0.363396,"scheduling_latency_s":0.363418,"redis_calls":13,"redis_duration_s":0.003123,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015161,"completed_at":"2026-06-29T22:08:06.601Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.873196] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0eb9ad6e459820a31bc32e7","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0eb9ad6e459820a31bc32e7: start","job_status":"start","queue_duration_s":0.37989,"scheduling_latency_s":0.379911} vm-test-run-nixbot-gitlab> gitlab # [ 190.874506] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0eb9ad6e459820a31bc32e7","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0eb9ad6e459820a31bc32e7: done: 0.015217 sec","job_status":"done","queue_duration_s":0.37989,"scheduling_latency_s":0.379911,"redis_calls":13,"redis_duration_s":0.003073,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015217,"completed_at":"2026-06-29T22:08:06.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.945051] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f8fbf61cb46534e928cde4a","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f8fbf61cb46534e928cde4a: start","job_status":"start","queue_duration_s":0.396475,"scheduling_latency_s":0.396496} vm-test-run-nixbot-gitlab> gitlab # [ 190.946402] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f8fbf61cb46534e928cde4a","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f8fbf61cb46534e928cde4a: done: 0.014785 sec","job_status":"done","queue_duration_s":0.396475,"scheduling_latency_s":0.396496,"redis_calls":13,"redis_duration_s":0.00286,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014785,"completed_at":"2026-06-29T22:08:06.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.041391] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cebc569f92209c81115fa1fb","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cebc569f92209c81115fa1fb: start","job_status":"start","queue_duration_s":0.41255,"scheduling_latency_s":0.412573} vm-test-run-nixbot-gitlab> gitlab # [ 191.042823] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cebc569f92209c81115fa1fb","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cebc569f92209c81115fa1fb: done: 0.014765 sec","job_status":"done","queue_duration_s":0.41255,"scheduling_latency_s":0.412573,"redis_calls":13,"redis_duration_s":0.002923,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014765,"completed_at":"2026-06-29T22:08:06.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.119950] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2af53dcca07f936e32f3a686","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2af53dcca07f936e32f3a686: start","job_status":"start","queue_duration_s":0.428571,"scheduling_latency_s":0.428592} vm-test-run-nixbot-gitlab> gitlab # [ 191.121334] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2af53dcca07f936e32f3a686","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2af53dcca07f936e32f3a686: done: 0.015107 sec","job_status":"done","queue_duration_s":0.428571,"scheduling_latency_s":0.428592,"redis_calls":13,"redis_duration_s":0.003316,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002395,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015107,"completed_at":"2026-06-29T22:08:06.666Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.151375] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77642ac2fe13acdca61da594","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77642ac2fe13acdca61da594: start","job_status":"start","queue_duration_s":0.44498,"scheduling_latency_s":0.445001} vm-test-run-nixbot-gitlab> gitlab # [ 191.231451] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77642ac2fe13acdca61da594","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77642ac2fe13acdca61da594: done: 0.015833 sec","job_status":"done","queue_duration_s":0.44498,"scheduling_latency_s":0.445001,"redis_calls":13,"redis_duration_s":0.003049,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015833,"completed_at":"2026-06-29T22:08:06.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.233166] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a8ec782a2cc49c31650dbab","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a8ec782a2cc49c31650dbab: start","job_status":"start","queue_duration_s":0.462145,"scheduling_latency_s":0.462165} vm-test-run-nixbot-gitlab> gitlab # [ 191.318458] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a8ec782a2cc49c31650dbab","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a8ec782a2cc49c31650dbab: done: 0.014917 sec","job_status":"done","queue_duration_s":0.462145,"scheduling_latency_s":0.462165,"redis_calls":13,"redis_duration_s":0.003033,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014917,"completed_at":"2026-06-29T22:08:06.700Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.320202] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c492cf53de84ac899856f2c","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c492cf53de84ac899856f2c: start","job_status":"start","queue_duration_s":0.478506,"scheduling_latency_s":0.478528} vm-test-run-nixbot-gitlab> gitlab # [ 191.396436] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c492cf53de84ac899856f2c","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c492cf53de84ac899856f2c: done: 0.014996 sec","job_status":"done","queue_duration_s":0.478506,"scheduling_latency_s":0.478528,"redis_calls":13,"redis_duration_s":0.00321,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002196,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014996,"completed_at":"2026-06-29T22:08:06.716Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.398860] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53f91d627cccd3c2551f97d1","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53f91d627cccd3c2551f97d1: start","job_status":"start","queue_duration_s":0.494799,"scheduling_latency_s":0.494822} vm-test-run-nixbot-gitlab> gitlab # [ 191.468267] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53f91d627cccd3c2551f97d1","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53f91d627cccd3c2551f97d1: done: 0.014785 sec","job_status":"done","queue_duration_s":0.494799,"scheduling_latency_s":0.494822,"redis_calls":13,"redis_duration_s":0.002935,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014785,"completed_at":"2026-06-29T22:08:06.732Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.474279] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4b810881e7a7c08c63b25f6","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4b810881e7a7c08c63b25f6: start","job_status":"start","queue_duration_s":0.51086,"scheduling_latency_s":0.510882} vm-test-run-nixbot-gitlab> gitlab # [ 191.494416] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4b810881e7a7c08c63b25f6","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4b810881e7a7c08c63b25f6: done: 0.014705 sec","job_status":"done","queue_duration_s":0.51086,"scheduling_latency_s":0.510882,"redis_calls":13,"redis_duration_s":0.0028870000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014705,"completed_at":"2026-06-29T22:08:06.748Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.557828] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2252fb79deffd9569686ffa9","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2252fb79deffd9569686ffa9: start","job_status":"start","queue_duration_s":0.526828,"scheduling_latency_s":0.526849} vm-test-run-nixbot-gitlab> gitlab # [ 191.577828] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2252fb79deffd9569686ffa9","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2252fb79deffd9569686ffa9: done: 0.015085 sec","job_status":"done","queue_duration_s":0.526828,"scheduling_latency_s":0.526849,"redis_calls":13,"redis_duration_s":0.003293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002333,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015085,"completed_at":"2026-06-29T22:08:06.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.662169] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4444ab2f52e84e5c71e8bbc","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4444ab2f52e84e5c71e8bbc: start","job_status":"start","queue_duration_s":0.543215,"scheduling_latency_s":0.543235} vm-test-run-nixbot-gitlab> gitlab # [ 191.663229] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4444ab2f52e84e5c71e8bbc","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4444ab2f52e84e5c71e8bbc: done: 0.015947 sec","job_status":"done","queue_duration_s":0.543215,"scheduling_latency_s":0.543235,"redis_calls":13,"redis_duration_s":0.004021,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015947,"completed_at":"2026-06-29T22:08:06.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.732388] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b88ee98549c8e252fb3a5d8","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b88ee98549c8e252fb3a5d8: start","job_status":"start","queue_duration_s":0.560497,"scheduling_latency_s":0.560517} vm-test-run-nixbot-gitlab> gitlab # [ 191.803113] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b88ee98549c8e252fb3a5d8","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b88ee98549c8e252fb3a5d8: done: 0.014762 sec","job_status":"done","queue_duration_s":0.560497,"scheduling_latency_s":0.560517,"redis_calls":13,"redis_duration_s":0.0028840000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014762,"completed_at":"2026-06-29T22:08:06.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.805140] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6a5afdc3f64a45bfd3b87db","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6a5afdc3f64a45bfd3b87db: start","job_status":"start","queue_duration_s":0.576561,"scheduling_latency_s":0.576583} vm-test-run-nixbot-gitlab> gitlab # [ 191.819182] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6a5afdc3f64a45bfd3b87db","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6a5afdc3f64a45bfd3b87db: done: 0.014723 sec","job_status":"done","queue_duration_s":0.576561,"scheduling_latency_s":0.576583,"redis_calls":13,"redis_duration_s":0.0029179999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014723,"completed_at":"2026-06-29T22:08:06.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.895066] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"152650d0603879e1dd01b68b","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-152650d0603879e1dd01b68b: start","job_status":"start","queue_duration_s":0.592543,"scheduling_latency_s":0.592565} vm-test-run-nixbot-gitlab> gitlab # [ 191.900095] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"152650d0603879e1dd01b68b","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-152650d0603879e1dd01b68b: done: 0.014602 sec","job_status":"done","queue_duration_s":0.592543,"scheduling_latency_s":0.592565,"redis_calls":13,"redis_duration_s":0.002852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014602,"completed_at":"2026-06-29T22:08:06.830Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.963503] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dac23962748b6c570c8dfe4c","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dac23962748b6c570c8dfe4c: start","job_status":"start","queue_duration_s":0.608461,"scheduling_latency_s":0.608483} vm-test-run-nixbot-gitlab> gitlab # [ 191.983510] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dac23962748b6c570c8dfe4c","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dac23962748b6c570c8dfe4c: done: 0.014659 sec","job_status":"done","queue_duration_s":0.608461,"scheduling_latency_s":0.608483,"redis_calls":13,"redis_duration_s":0.002859,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014659,"completed_at":"2026-06-29T22:08:06.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.052489] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"117d542ad95e04053ca58167","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-117d542ad95e04053ca58167: start","job_status":"start","queue_duration_s":0.62446,"scheduling_latency_s":0.624481} vm-test-run-nixbot-gitlab> gitlab # [ 192.064075] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"117d542ad95e04053ca58167","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-117d542ad95e04053ca58167: done: 0.025458 sec","job_status":"done","queue_duration_s":0.62446,"scheduling_latency_s":0.624481,"redis_calls":13,"redis_duration_s":0.009651,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00325,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006401,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025458,"completed_at":"2026-06-29T22:08:06.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.144120] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23a225ba1010bc90b8c8f0fa","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23a225ba1010bc90b8c8f0fa: start","job_status":"start","queue_duration_s":0.651312,"scheduling_latency_s":0.651334} vm-test-run-nixbot-gitlab> gitlab # [ 192.145120] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23a225ba1010bc90b8c8f0fa","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23a225ba1010bc90b8c8f0fa: done: 0.015073 sec","job_status":"done","queue_duration_s":0.651312,"scheduling_latency_s":0.651334,"redis_calls":13,"redis_duration_s":0.00296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015073,"completed_at":"2026-06-29T22:08:06.889Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.215363] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"378fe5a6274d8058e8338937","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-378fe5a6274d8058e8338937: start","job_status":"start","queue_duration_s":0.667674,"scheduling_latency_s":0.667697} vm-test-run-nixbot-gitlab> gitlab # [ 192.224864] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"378fe5a6274d8058e8338937","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-378fe5a6274d8058e8338937: done: 0.014718 sec","job_status":"done","queue_duration_s":0.667674,"scheduling_latency_s":0.667697,"redis_calls":13,"redis_duration_s":0.002938,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014718,"completed_at":"2026-06-29T22:08:06.905Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.303798] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f658ff067c7eac7b0853174","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f658ff067c7eac7b0853174: start","job_status":"start","queue_duration_s":0.683674,"scheduling_latency_s":0.683696} vm-test-run-nixbot-gitlab> gitlab # [ 192.306173] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f658ff067c7eac7b0853174","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f658ff067c7eac7b0853174: done: 0.014635 sec","job_status":"done","queue_duration_s":0.683674,"scheduling_latency_s":0.683696,"redis_calls":13,"redis_duration_s":0.002874,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014635,"completed_at":"2026-06-29T22:08:06.921Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.375264] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50d0612ab393ff0a9c1339bf","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50d0612ab393ff0a9c1339bf: start","job_status":"start","queue_duration_s":0.699593,"scheduling_latency_s":0.699614} vm-test-run-nixbot-gitlab> gitlab # [ 192.386974] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50d0612ab393ff0a9c1339bf","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50d0612ab393ff0a9c1339bf: done: 0.014693 sec","job_status":"done","queue_duration_s":0.699593,"scheduling_latency_s":0.699614,"redis_calls":13,"redis_duration_s":0.0029309999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014693,"completed_at":"2026-06-29T22:08:06.937Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.450220] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"192b52c2761dbfc2acf0d8e1","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-192b52c2761dbfc2acf0d8e1: start","job_status":"start","queue_duration_s":0.715558,"scheduling_latency_s":0.715579} vm-test-run-nixbot-gitlab> gitlab # [ 192.517567] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"192b52c2761dbfc2acf0d8e1","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-192b52c2761dbfc2acf0d8e1: done: 0.014969 sec","job_status":"done","queue_duration_s":0.715558,"scheduling_latency_s":0.715579,"redis_calls":13,"redis_duration_s":0.003087,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014969,"completed_at":"2026-06-29T22:08:06.953Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.528796] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"634f7294e13a122861c00298","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-634f7294e13a122861c00298: start","job_status":"start","queue_duration_s":0.731864,"scheduling_latency_s":0.731886} vm-test-run-nixbot-gitlab> gitlab # [ 192.548365] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"634f7294e13a122861c00298","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-634f7294e13a122861c00298: done: 0.018563 sec","job_status":"done","queue_duration_s":0.731864,"scheduling_latency_s":0.731886,"redis_calls":13,"redis_duration_s":0.003358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002336,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018563,"completed_at":"2026-06-29T22:08:06.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.628097] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19cadc8a5bd20232cfc07211","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19cadc8a5bd20232cfc07211: start","job_status":"start","queue_duration_s":0.751782,"scheduling_latency_s":0.751805} vm-test-run-nixbot-gitlab> gitlab # [ 192.629457] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19cadc8a5bd20232cfc07211","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19cadc8a5bd20232cfc07211: done: 0.014994 sec","job_status":"done","queue_duration_s":0.751782,"scheduling_latency_s":0.751805,"redis_calls":13,"redis_duration_s":0.00289,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014994,"completed_at":"2026-06-29T22:08:06.990Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.722284] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:06.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1002bf397b230a86ba2e622c","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1002bf397b230a86ba2e622c: start","job_status":"start","queue_duration_s":0.768041,"scheduling_latency_s":0.768061} vm-test-run-nixbot-gitlab> gitlab # [ 192.723918] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1002bf397b230a86ba2e622c","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1002bf397b230a86ba2e622c: done: 0.01477 sec","job_status":"done","queue_duration_s":0.768041,"scheduling_latency_s":0.768061,"redis_calls":13,"redis_duration_s":0.002823,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01477,"completed_at":"2026-06-29T22:08:07.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.795148] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab81bef2bb70272a8dbb2e88","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab81bef2bb70272a8dbb2e88: start","job_status":"start","queue_duration_s":0.784097,"scheduling_latency_s":0.784118} vm-test-run-nixbot-gitlab> gitlab # [ 192.796450] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab81bef2bb70272a8dbb2e88","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab81bef2bb70272a8dbb2e88: done: 0.050395 sec","job_status":"done","queue_duration_s":0.784097,"scheduling_latency_s":0.784118,"redis_calls":13,"redis_duration_s":0.002945,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.050395,"completed_at":"2026-06-29T22:08:07.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.867056] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eabc5f9bfecba175c372821c","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eabc5f9bfecba175c372821c: start","job_status":"start","queue_duration_s":0.836072,"scheduling_latency_s":0.836097} vm-test-run-nixbot-gitlab> gitlab # [ 192.868357] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eabc5f9bfecba175c372821c","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eabc5f9bfecba175c372821c: done: 0.016486 sec","job_status":"done","queue_duration_s":0.836072,"scheduling_latency_s":0.836097,"redis_calls":13,"redis_duration_s":0.0031590000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001159,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016486,"completed_at":"2026-06-29T22:08:07.075Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.941281] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"313d373476c80e81a98092e5","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-313d373476c80e81a98092e5: start","job_status":"start","queue_duration_s":0.853813,"scheduling_latency_s":0.853834} vm-test-run-nixbot-gitlab> gitlab # [ 192.948052] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"313d373476c80e81a98092e5","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-313d373476c80e81a98092e5: done: 0.014785 sec","job_status":"done","queue_duration_s":0.853813,"scheduling_latency_s":0.853834,"redis_calls":13,"redis_duration_s":0.0028970000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014785,"completed_at":"2026-06-29T22:08:07.091Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.056630] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e3a76dcb256253d73c44ede","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e3a76dcb256253d73c44ede: start","job_status":"start","queue_duration_s":0.86986,"scheduling_latency_s":0.869881} vm-test-run-nixbot-gitlab> gitlab # [ 193.058153] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e3a76dcb256253d73c44ede","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e3a76dcb256253d73c44ede: done: 0.015076 sec","job_status":"done","queue_duration_s":0.86986,"scheduling_latency_s":0.869881,"redis_calls":13,"redis_duration_s":0.0029130000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015076,"completed_at":"2026-06-29T22:08:07.108Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.127265] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f15b9f516a0aec277c4df373","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f15b9f516a0aec277c4df373: start","job_status":"start","queue_duration_s":0.886607,"scheduling_latency_s":0.886628} vm-test-run-nixbot-gitlab> gitlab # [ 193.128824] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f15b9f516a0aec277c4df373","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f15b9f516a0aec277c4df373: done: 0.015341 sec","job_status":"done","queue_duration_s":0.886607,"scheduling_latency_s":0.886628,"redis_calls":13,"redis_duration_s":0.0031929999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001122,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015341,"completed_at":"2026-06-29T22:08:07.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.205430] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b54f84e8865d8d6237b772d4","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b54f84e8865d8d6237b772d4: start","job_status":"start","queue_duration_s":0.903343,"scheduling_latency_s":0.903364} vm-test-run-nixbot-gitlab> gitlab # [ 193.206899] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b54f84e8865d8d6237b772d4","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b54f84e8865d8d6237b772d4: done: 0.020805 sec","job_status":"done","queue_duration_s":0.903343,"scheduling_latency_s":0.903364,"redis_calls":13,"redis_duration_s":0.004535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00112,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003415,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020805,"completed_at":"2026-06-29T22:08:07.147Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.293082] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a20c57acf1e05723c17882f","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a20c57acf1e05723c17882f: start","job_status":"start","queue_duration_s":0.925929,"scheduling_latency_s":0.925962} vm-test-run-nixbot-gitlab> gitlab # [ 193.294500] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a20c57acf1e05723c17882f","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a20c57acf1e05723c17882f: done: 0.018291 sec","job_status":"done","queue_duration_s":0.925929,"scheduling_latency_s":0.925962,"redis_calls":13,"redis_duration_s":0.003117,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018291,"completed_at":"2026-06-29T22:08:07.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.365378] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6db21f35999455f9fde33e1a","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6db21f35999455f9fde33e1a: start","job_status":"start","queue_duration_s":0.946086,"scheduling_latency_s":0.946112} vm-test-run-nixbot-gitlab> gitlab # [ 193.366403] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6db21f35999455f9fde33e1a","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6db21f35999455f9fde33e1a: done: 0.015577 sec","job_status":"done","queue_duration_s":0.946086,"scheduling_latency_s":0.946112,"redis_calls":13,"redis_duration_s":0.0029179999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015577,"completed_at":"2026-06-29T22:08:07.184Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.441919] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2041272bcc7389d6722c8afb","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2041272bcc7389d6722c8afb: start","job_status":"start","queue_duration_s":0.962917,"scheduling_latency_s":0.962937} vm-test-run-nixbot-gitlab> gitlab # [ 193.443235] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2041272bcc7389d6722c8afb","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2041272bcc7389d6722c8afb: done: 0.017642 sec","job_status":"done","queue_duration_s":0.962917,"scheduling_latency_s":0.962937,"redis_calls":13,"redis_duration_s":0.003156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017642,"completed_at":"2026-06-29T22:08:07.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.533574] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"326c09eb44f6a96f208964d1","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-326c09eb44f6a96f208964d1: start","job_status":"start","queue_duration_s":0.982,"scheduling_latency_s":0.982021} vm-test-run-nixbot-gitlab> gitlab # [ 193.534843] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"326c09eb44f6a96f208964d1","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-326c09eb44f6a96f208964d1: done: 0.015858 sec","job_status":"done","queue_duration_s":0.982,"scheduling_latency_s":0.982021,"redis_calls":13,"redis_duration_s":0.002993,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015858,"completed_at":"2026-06-29T22:08:07.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.614831] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"772e28bbe2e0aa3f8405efcc","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-772e28bbe2e0aa3f8405efcc: start","job_status":"start","queue_duration_s":0.999128,"scheduling_latency_s":0.999151} vm-test-run-nixbot-gitlab> gitlab # [ 193.618274] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"772e28bbe2e0aa3f8405efcc","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-772e28bbe2e0aa3f8405efcc: done: 0.015199 sec","job_status":"done","queue_duration_s":0.999128,"scheduling_latency_s":0.999151,"redis_calls":13,"redis_duration_s":0.002872,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015199,"completed_at":"2026-06-29T22:08:07.237Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.650181] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"782705ace62b21fb996308fd","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-782705ace62b21fb996308fd: start","job_status":"start","queue_duration_s":1.017414,"scheduling_latency_s":1.017445} vm-test-run-nixbot-gitlab> gitlab # [ 193.730084] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"782705ace62b21fb996308fd","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-782705ace62b21fb996308fd: done: 0.01518 sec","job_status":"done","queue_duration_s":1.017414,"scheduling_latency_s":1.017445,"redis_calls":13,"redis_duration_s":0.0029059999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01518,"completed_at":"2026-06-29T22:08:07.255Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.732047] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69c72900a2a3be8b54d16586","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69c72900a2a3be8b54d16586: start","job_status":"start","queue_duration_s":1.033861,"scheduling_latency_s":1.033882} vm-test-run-nixbot-gitlab> gitlab # [ 193.748938] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69c72900a2a3be8b54d16586","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69c72900a2a3be8b54d16586: done: 0.015506 sec","job_status":"done","queue_duration_s":1.033861,"scheduling_latency_s":1.033882,"redis_calls":13,"redis_duration_s":0.0032110000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002285,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015506,"completed_at":"2026-06-29T22:08:07.272Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.868884] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eac602dd385c01fee4ff89ed","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eac602dd385c01fee4ff89ed: start","job_status":"start","queue_duration_s":1.050652,"scheduling_latency_s":1.050672} vm-test-run-nixbot-gitlab> gitlab # [ 193.870334] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eac602dd385c01fee4ff89ed","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eac602dd385c01fee4ff89ed: done: 0.015384 sec","job_status":"done","queue_duration_s":1.050652,"scheduling_latency_s":1.050672,"redis_calls":13,"redis_duration_s":0.003123,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015384,"completed_at":"2026-06-29T22:08:07.289Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.889757] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2966997fb5996a5626cb66af","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2966997fb5996a5626cb66af: start","job_status":"start","queue_duration_s":1.068791,"scheduling_latency_s":1.068813} vm-test-run-nixbot-gitlab> gitlab # [ 193.968102] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2966997fb5996a5626cb66af","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2966997fb5996a5626cb66af: done: 0.015481 sec","job_status":"done","queue_duration_s":1.068791,"scheduling_latency_s":1.068813,"redis_calls":13,"redis_duration_s":0.0030489999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015481,"completed_at":"2026-06-29T22:08:07.307Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.970091] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bc34b720c850e551e1fbca2","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bc34b720c850e551e1fbca2: start","job_status":"start","queue_duration_s":1.085545,"scheduling_latency_s":1.085568} vm-test-run-nixbot-gitlab> gitlab # [ 193.988355] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bc34b720c850e551e1fbca2","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bc34b720c850e551e1fbca2: done: 0.015133 sec","job_status":"done","queue_duration_s":1.085545,"scheduling_latency_s":1.085568,"redis_calls":13,"redis_duration_s":0.0028650000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015133,"completed_at":"2026-06-29T22:08:07.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.070181] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.325Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c35e85d3ff4b29482a3c97f1","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c35e85d3ff4b29482a3c97f1: start","job_status":"start","queue_duration_s":1.101922,"scheduling_latency_s":1.101943} vm-test-run-nixbot-gitlab> gitlab # [ 194.071422] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c35e85d3ff4b29482a3c97f1","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c35e85d3ff4b29482a3c97f1: done: 0.015073 sec","job_status":"done","queue_duration_s":1.101922,"scheduling_latency_s":1.101943,"redis_calls":13,"redis_duration_s":0.002948,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015073,"completed_at":"2026-06-29T22:08:07.340Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.140701] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afc3e74a44e6664d1abfbd51","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afc3e74a44e6664d1abfbd51: start","job_status":"start","queue_duration_s":1.118303,"scheduling_latency_s":1.118323} vm-test-run-nixbot-gitlab> gitlab # [ 194.207694] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afc3e74a44e6664d1abfbd51","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afc3e74a44e6664d1abfbd51: done: 0.01502 sec","job_status":"done","queue_duration_s":1.118303,"scheduling_latency_s":1.118323,"redis_calls":13,"redis_duration_s":0.0028710000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01502,"completed_at":"2026-06-29T22:08:07.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.209912] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32f69688d4f218fdcf60afdc","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32f69688d4f218fdcf60afdc: start","job_status":"start","queue_duration_s":1.134592,"scheduling_latency_s":1.134611} vm-test-run-nixbot-gitlab> gitlab # [ 194.278498] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32f69688d4f218fdcf60afdc","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32f69688d4f218fdcf60afdc: done: 0.016495 sec","job_status":"done","queue_duration_s":1.134592,"scheduling_latency_s":1.134611,"redis_calls":13,"redis_duration_s":0.0034080000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002426,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016495,"completed_at":"2026-06-29T22:08:07.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.345802] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf2a13bc5a0956961f1e6890","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf2a13bc5a0956961f1e6890: start","job_status":"start","queue_duration_s":1.152459,"scheduling_latency_s":1.15248} vm-test-run-nixbot-gitlab> gitlab # [ 194.347162] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf2a13bc5a0956961f1e6890","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf2a13bc5a0956961f1e6890: done: 0.015974 sec","job_status":"done","queue_duration_s":1.152459,"scheduling_latency_s":1.15248,"redis_calls":13,"redis_duration_s":0.003407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002508,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015974,"completed_at":"2026-06-29T22:08:07.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.419407] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1f5bb76c3ac5cd6c80d8126","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1f5bb76c3ac5cd6c80d8126: start","job_status":"start","queue_duration_s":1.169718,"scheduling_latency_s":1.169741} vm-test-run-nixbot-gitlab> gitlab # [ 194.420395] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1f5bb76c3ac5cd6c80d8126","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1f5bb76c3ac5cd6c80d8126: done: 0.015231 sec","job_status":"done","queue_duration_s":1.169718,"scheduling_latency_s":1.169741,"redis_calls":13,"redis_duration_s":0.003468,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001431,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015231,"completed_at":"2026-06-29T22:08:07.408Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.493320] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9822c432882e7fb0e79fbdf6","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9822c432882e7fb0e79fbdf6: start","job_status":"start","queue_duration_s":1.186229,"scheduling_latency_s":1.18625} vm-test-run-nixbot-gitlab> gitlab # [ 194.494221] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9822c432882e7fb0e79fbdf6","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9822c432882e7fb0e79fbdf6: done: 0.015105 sec","job_status":"done","queue_duration_s":1.186229,"scheduling_latency_s":1.18625,"redis_calls":13,"redis_duration_s":0.002891,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015105,"completed_at":"2026-06-29T22:08:07.424Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.570407] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db40007c8901d42b06425327","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db40007c8901d42b06425327: start","job_status":"start","queue_duration_s":1.203135,"scheduling_latency_s":1.203156} vm-test-run-nixbot-gitlab> gitlab # [ 194.571911] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db40007c8901d42b06425327","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db40007c8901d42b06425327: done: 0.015154 sec","job_status":"done","queue_duration_s":1.203135,"scheduling_latency_s":1.203156,"redis_calls":13,"redis_duration_s":0.0028409999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015154,"completed_at":"2026-06-29T22:08:07.441Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.651386] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"034eda2ec03faee50264b94a","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-034eda2ec03faee50264b94a: start","job_status":"start","queue_duration_s":1.21956,"scheduling_latency_s":1.21958} vm-test-run-nixbot-gitlab> gitlab # [ 194.653096] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"034eda2ec03faee50264b94a","created_at":"2026-06-29T22:08:06.144Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-b8ca6214e4cb4c1f","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9fe07ae1153cab3bcb7bdf9640a14a99","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.223Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-034eda2ec03faee50264b94a: done: 0.015461 sec","job_status":"done","queue_duration_s":1.21956,"scheduling_latency_s":1.21958,"redis_calls":13,"redis_duration_s":0.0030280000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015461,"completed_at":"2026-06-29T22:08:07.458Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.732299] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.459Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bd0798cae3b769482065eb2b","created_at":"2026-06-29T22:08:06.401Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-448adc72d396445c","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.408Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bd0798cae3b769482065eb2b: start","job_status":"start","queue_duration_s":1.050785,"scheduling_latency_s":1.050814} vm-test-run-nixbot-gitlab> gitlab # [ 194.733753] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.471Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 194.735087] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.477Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bd0798cae3b769482065eb2b","created_at":"2026-06-29T22:08:06.401Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-448adc72d396445c","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.408Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bd0798cae3b769482065eb2b: done: 0.017666 sec","job_status":"done","queue_duration_s":1.050785,"scheduling_latency_s":1.050814,"redis_calls":10,"redis_duration_s":0.003203,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002191,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017666,"completed_at":"2026-06-29T22:08:07.477Z","load_balancing_strategy":"primary","db_duration_s":0.002375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.760249] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.478Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"751860c28b9cf00794d40d6e","created_at":"2026-06-29T22:08:06.401Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-448adc72d396445c","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.408Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-751860c28b9cf00794d40d6e: start","job_status":"start","queue_duration_s":1.069772,"scheduling_latency_s":1.069802} vm-test-run-nixbot-gitlab> gitlab # [ 194.780217] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.488Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 194.782926] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.495Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"751860c28b9cf00794d40d6e","created_at":"2026-06-29T22:08:06.401Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-448adc72d396445c","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.408Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-751860c28b9cf00794d40d6e: done: 0.016431 sec","job_status":"done","queue_duration_s":1.069772,"scheduling_latency_s":1.069802,"redis_calls":10,"redis_duration_s":0.002384,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00141,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016431,"completed_at":"2026-06-29T22:08:07.495Z","load_balancing_strategy":"primary","db_duration_s":0.00158,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.900137] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.496Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1db14dcbb992e513c3165b91","created_at":"2026-06-29T22:08:06.401Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-448adc72d396445c","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.408Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1db14dcbb992e513c3165b91: start","job_status":"start","queue_duration_s":1.087576,"scheduling_latency_s":1.087598} vm-test-run-nixbot-gitlab> gitlab # [ 194.901428] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.506Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 194.902346] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.512Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1db14dcbb992e513c3165b91","created_at":"2026-06-29T22:08:06.401Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-448adc72d396445c","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.408Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1db14dcbb992e513c3165b91: done: 0.015644 sec","job_status":"done","queue_duration_s":1.087576,"scheduling_latency_s":1.087598,"redis_calls":10,"redis_duration_s":0.002338,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001411,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015644,"completed_at":"2026-06-29T22:08:07.512Z","load_balancing_strategy":"primary","db_duration_s":0.001628,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 194.927267] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.513Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"01d9b1e4f4f17829cdf3ce51","created_at":"2026-06-29T22:08:06.401Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-448adc72d396445c","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.408Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-01d9b1e4f4f17829cdf3ce51: start","job_status":"start","queue_duration_s":1.104436,"scheduling_latency_s":1.104456} vm-test-run-nixbot-gitlab> gitlab # [ 194.947216] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.523Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 194.950066] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:07.529Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"01d9b1e4f4f17829cdf3ce51","created_at":"2026-06-29T22:08:06.401Z","trace_propagation_headers":{"sentry-trace":"22fa4021bf8c40fd86eb20f4b6cac669-448adc72d396445c","baggage":"sentry-trace_id=22fa4021bf8c40fd86eb20f4b6cac669,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c07d7a4d937e914d38185cf1aea96036","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:06.408Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-01d9b1e4f4f17829cdf3ce51: done: 0.016333 sec","job_status":"done","queue_duration_s":1.104436,"scheduling_latency_s":1.104456,"redis_calls":10,"redis_duration_s":0.002529,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001557,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016333,"completed_at":"2026-06-29T22:08:07.529Z","load_balancing_strategy":"primary","db_duration_s":0.002373,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 195.342768] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 5171746621 wd_nsec: 5171744036 vm-test-run-nixbot-gitlab> gitlab # [ 198.867893] runuser[2735]: pam_unix(runuser:session): session closed for user gitlab vm-test-run-nixbot-gitlab> gitlab # [ 198.875683] setup-gitlab-start[2957]: ++ jq -r .token vm-test-run-nixbot-gitlab> gitlab # [ 198.877567] setup-gitlab-start[2958]: +++ date -d '+7 days' +%F vm-test-run-nixbot-gitlab> gitlab # [ 198.880934] setup-gitlab-start[2956]: ++ curl -fs -X POST -H 'Authorization: Bearer 4a2bb711b1be83cc7d1cf0e2d21072cc2e867617e138b17e83dcea2d5d241fac' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-06"}' http://gitlab/api/v4/users/1/personal_access_tokens vm-test-run-nixbot-gitlab> gitlab # [ 199.096811] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:16.002Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"4cada36912af316a0209476e","created_at":"2026-06-29T22:08:15.968Z","trace_propagation_headers":{"sentry-trace":"b4ac1def9b584470b63b03b834cd904b-1d2c1297a89d4fb2","baggage":"sentry-trace_id=b4ac1def9b584470b63b03b834cd904b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT2XQK6S2FDP8H5R5C3VM","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:1aaa7278415282300418518b09ab8b598ab9223c072a7d0cc495005e326e8d8b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:15.992Z","job_size_bytes":451,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-4cada36912af316a0209476e: start","job_status":"start","queue_duration_s":0.010596,"scheduling_latency_s":0.010649} vm-test-run-nixbot-gitlab> gitlab # [ 199.103419] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:16 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 209 vm-test-run-nixbot-gitlab> gitlab # [ 199.163241] setup-gitlab-start[2697]: + TOKEN=glpat-yU0KOqxn2pMTlPk2GGPVdW86MQp1OjEH.01.0w0ra882h vm-test-run-nixbot-gitlab> gitlab # [ 199.164531] setup-gitlab-start[2697]: + test -n glpat-yU0KOqxn2pMTlPk2GGPVdW86MQp1OjEH.01.0w0ra882h vm-test-run-nixbot-gitlab> gitlab # [ 199.166161] setup-gitlab-start[2697]: + test glpat-yU0KOqxn2pMTlPk2GGPVdW86MQp1OjEH.01.0w0ra882h '!=' null vm-test-run-nixbot-gitlab> gitlab # [ 199.167546] setup-gitlab-start[2697]: + mkdir -p /var/lib/secrets vm-test-run-nixbot-gitlab> gitlab # [ 199.171208] setup-gitlab-start[2697]: + echo glpat-yU0KOqxn2pMTlPk2GGPVdW86MQp1OjEH.01.0w0ra882h vm-test-run-nixbot-gitlab> gitlab # [ 199.172380] setup-gitlab-start[2697]: + chmod 644 /var/lib/secrets/gitlab-token vm-test-run-nixbot-gitlab> gitlab # [ 199.176753] setup-gitlab-start[2697]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-yU0KOqxn2pMTlPk2GGPVdW86MQp1OjEH.01.0w0ra882h' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects vm-test-run-nixbot-gitlab> gitlab # [ 199.214368] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T22:08:16.120Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"4cada36912af316a0209476e","created_at":"2026-06-29T22:08:15.968Z","trace_propagation_headers":{"sentry-trace":"b4ac1def9b584470b63b03b834cd904b-1d2c1297a89d4fb2","baggage":"sentry-trace_id=b4ac1def9b584470b63b03b834cd904b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT2XQK6S2FDP8H5R5C3VM","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:1aaa7278415282300418518b09ab8b598ab9223c072a7d0cc495005e326e8d8b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:15.992Z","job_size_bytes":451,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-4cada36912af316a0209476e: fail: 0.117994 sec","job_status":"fail","queue_duration_s":0.010596,"scheduling_latency_s":0.010649,"redis_calls":11,"redis_duration_s":0.005453,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001891,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002736,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000826,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000369879,"duration_s":0.117994,"completed_at":"2026-06-29T22:08:16.120Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.016538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 200.324210] gitaly[1382]: time="2026-06-29T22:08:17.229Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.229" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.325816] gitaly[1382]: time="2026-06-29T22:08:17.230Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.229" grpc.time_ms=0.842 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.475347] gitaly[1382]: time="2026-06-29T22:08:17.382Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.160" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.382" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.514423] gitaly[1382]: time="2026-06-29T22:08:17.382Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.160" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.382" grpc.time_ms=0.598 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.562211] gitaly[1382]: time="2026-06-29T22:08:17.469Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z diskcache=bbef4414-0909-4a27-8642-c8350994cb90 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T22:08:27.208" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.429" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.583051] gitaly[1382]: time="2026-06-29T22:08:17.470Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=32 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T22:08:27.208" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.429" grpc.time_ms=39.879 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.583915] gitaly[1382]: time="2026-06-29T22:08:17.490Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.268" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.489" pid=1382 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.584964] gitaly[1382]: time="2026-06-29T22:08:17.490Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.268" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.489" grpc.time_ms=0.969 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.662718] gitaly[1382]: time="2026-06-29T22:08:17.569Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T22:08:27.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.555" grpc.time_ms=14.263 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.701396] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:17.607Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"472e3c4ed2488a626db04ea1","created_at":"2026-06-29T22:08:17.585Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:17.604Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-472e3c4ed2488a626db04ea1: start","job_status":"start","queue_duration_s":0.00304,"scheduling_latency_s":0.003102} vm-test-run-nixbot-gitlab> gitlab # [ 200.730136] gitaly[1382]: time="2026-06-29T22:08:17.637Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.086" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.637" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.731424] gitaly[1382]: time="2026-06-29T22:08:17.637Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.086" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.637" grpc.time_ms=0.305 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.774428] gitaly[1382]: time="2026-06-29T22:08:17.681Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z diskcache=b5377ae3-288c-4daf-9527-8d7d69ccccb2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T22:08:27.096" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.647" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.776949] gitaly[1382]: time="2026-06-29T22:08:17.681Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=32 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T22:08:27.096" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.647" grpc.time_ms=34.596 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.786154] gitaly[1382]: time="2026-06-29T22:08:17.693Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.141" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.692" pid=1382 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.787522] gitaly[1382]: time="2026-06-29T22:08:17.693Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.141" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.692" grpc.time_ms=0.817 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.788880] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:17.693Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"472e3c4ed2488a626db04ea1","created_at":"2026-06-29T22:08:17.585Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:17.604Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-472e3c4ed2488a626db04ea1: done: 0.08625 sec","job_status":"done","queue_duration_s":0.00304,"scheduling_latency_s":0.003102,"redis_calls":10,"redis_duration_s":0.004843,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002696,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002147,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.050964857,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.08625,"completed_at":"2026-06-29T22:08:17.693Z","load_balancing_strategy":"primary","db_duration_s":0.01584,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 200.868553] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:17.696Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"32a2ee1fd7522eeda03d2d42","created_at":"2026-06-29T22:08:17.616Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:17.629Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-32a2ee1fd7522eeda03d2d42: start","job_status":"start","queue_duration_s":0.067022,"scheduling_latency_s":0.067083} vm-test-run-nixbot-gitlab> gitlab # [ 200.917622] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:17.824Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"32a2ee1fd7522eeda03d2d42","created_at":"2026-06-29T22:08:17.616Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:17.629Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-32a2ee1fd7522eeda03d2d42: done: 0.128648 sec","job_status":"done","queue_duration_s":0.067022,"scheduling_latency_s":0.067083,"redis_calls":14,"redis_duration_s":0.005377,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000758,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002844,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001775,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.098353003,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.128648,"completed_at":"2026-06-29T22:08:17.824Z","load_balancing_strategy":"primary","db_duration_s":0.052773,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 201.020058] gitaly[1382]: time="2026-06-29T22:08:17.927Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T22:08:27.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.918" grpc.time_ms=8.311 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 201.042423] gitaly[1382]: time="2026-06-29T22:08:17.949Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:27.038" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.948" grpc.time_ms=0.286 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 201.060534] gitaly[1382]: time="2026-06-29T22:08:17.964Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T22:08:27.051" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:17.962" grpc.time_ms=0.647 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 201.370779] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:18.277Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"8569d6e8714506ad8a4e8327","created_at":"2026-06-29T22:08:18.267Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:18.275Z","job_size_bytes":5,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-8569d6e8714506ad8a4e8327: start","job_status":"start","queue_duration_s":0.002347,"scheduling_latency_s":0.002394} vm-test-run-nixbot-gitlab> gitlab # [ 201.522827] gitaly[1382]: time="2026-06-29T22:08:18.426Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT36Y9JNGEJAKASC7DZ0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T22:08:28.197" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:18.417" grpc.time_ms=8.397 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 201.584995] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:18.491Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"8569d6e8714506ad8a4e8327","created_at":"2026-06-29T22:08:18.267Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:18.275Z","job_size_bytes":5,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-8569d6e8714506ad8a4e8327: done: 0.214477 sec","job_status":"done","queue_duration_s":0.002347,"scheduling_latency_s":0.002394,"redis_calls":12,"redis_duration_s":0.004899,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001682,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002565,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000652,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000434134,"duration_s":0.214477,"completed_at":"2026-06-29T22:08:18.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023317,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 201.658599] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:18.565Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"96958b19c0b56cf179cb1d9a","created_at":"2026-06-29T22:08:18.386Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:18.392Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-96958b19c0b56cf179cb1d9a: start","job_status":"start","queue_duration_s":0.172175,"scheduling_latency_s":0.172217} vm-test-run-nixbot-gitlab> gitlab # [ 201.869904] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:18 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 2680 vm-test-run-nixbot-gitlab> gitlab # [ 201.870718] setup-gitlab-start[2697]: + mkdir -p /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 201.874183] setup-gitlab-start[2697]: + cd /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 201.874925] setup-gitlab-start[2697]: + git init -b master vm-test-run-nixbot-gitlab> gitlab # [ 201.897531] setup-gitlab-start[2997]: Initialized empty Git repository in /tmp/test-flake/.git/ vm-test-run-nixbot-gitlab> gitlab # [ 201.898572] setup-gitlab-start[2697]: + git config user.name test vm-test-run-nixbot-gitlab> gitlab # [ 201.904114] setup-gitlab-start[2697]: + git config user.email test@example.com vm-test-run-nixbot-gitlab> gitlab # [ 201.907925] setup-gitlab-start[2697]: + cat vm-test-run-nixbot-gitlab> gitlab # [ 201.910395] setup-gitlab-start[2697]: + git add flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 201.914868] setup-gitlab-start[2697]: + git commit -m 'initial commit' vm-test-run-nixbot-gitlab> gitlab # [ 201.921823] setup-gitlab-start[3002]: [master (root-commit) 1ab2c6a] initial commit vm-test-run-nixbot-gitlab> gitlab # [ 201.922325] setup-gitlab-start[3002]: 1 file changed, 10 insertions(+) vm-test-run-nixbot-gitlab> gitlab # [ 201.923493] setup-gitlab-start[3002]: create mode 100644 flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 201.926316] setup-gitlab-start[2697]: + git remote add origin http://oauth2:glpat-yU0KOqxn2pMTlPk2GGPVdW86MQp1OjEH.01.0w0ra882h@gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 201.932929] setup-gitlab-start[2697]: + git push -u origin master vm-test-run-nixbot-gitlab> gitlab # [ 201.982290] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:18.888Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"96958b19c0b56cf179cb1d9a","created_at":"2026-06-29T22:08:18.386Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:18.392Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-96958b19c0b56cf179cb1d9a: done: 0.323798 sec","job_status":"done","queue_duration_s":0.172175,"scheduling_latency_s":0.172217,"redis_calls":15,"redis_duration_s":0.005178,"redis_read_bytes":141,"redis_write_bytes":1685,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.002743,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":945,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00185,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000585,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":37,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":37,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":36,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.229,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.229,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.197306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000420724,"duration_s":0.323798,"completed_at":"2026-06-29T22:08:18.888Z","load_balancing_strategy":"primary","db_duration_s":0.091639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 201.984845] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:18.890Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"129c36e8c4e588bf4a7e8b20","created_at":"2026-06-29T22:08:18.397Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:18.401Z","job_size_bytes":89,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-129c36e8c4e588bf4a7e8b20: start","job_status":"start","queue_duration_s":0.48841,"scheduling_latency_s":0.488434} vm-test-run-nixbot-gitlab> gitlab # [ 202.056320] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:18.952Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"129c36e8c4e588bf4a7e8b20","created_at":"2026-06-29T22:08:18.397Z","trace_propagation_headers":{"sentry-trace":"5312549079644e669fb4728a68eade30-13afd76016c944f6","baggage":"sentry-trace_id=5312549079644e669fb4728a68eade30,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT36Y9JNGEJAKASC7DZ0Z","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:18.401Z","job_size_bytes":89,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-129c36e8c4e588bf4a7e8b20: done: 0.061903 sec","job_status":"done","queue_duration_s":0.48841,"scheduling_latency_s":0.488434,"redis_calls":24,"redis_duration_s":0.005666,"redis_read_bytes":277,"redis_write_bytes":2951,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.00374,"redis_feature_flag_read_bytes":270,"redis_feature_flag_write_bytes":2127,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001464,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000409829,"duration_s":0.061903,"completed_at":"2026-06-29T22:08:18.952Z","load_balancing_strategy":"primary","db_duration_s":0.017677,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 202.366533] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:19 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 410 vm-test-run-nixbot-gitlab> gitlab # [ 202.497321] gitaly[1382]: time="2026-06-29T22:08:19.404Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6AA3PHZ4QV2Z7PQ3BKQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:19.386" grpc.time_ms=17.825 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 202.544783] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:19 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 176 vm-test-run-nixbot-gitlab> gitlab # [ 203.075741] gitaly[1382]: time="2026-06-29T22:08:19.982Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T22:08:29.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:19.967" grpc.time_ms=14.321 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.193065] gitaly[1382]: time="2026-06-29T22:08:20.100Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=326 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T22:08:50.089" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T22:08:20.090" grpc.time_ms=9.583 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.313169] gitaly[1382]: time="2026-06-29T22:08:20.220Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWAPT6GD89R0QK5WS2A245QM duration_ms=582ns method=POST pid=1382 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 203.359332] gitaly[1382]: time="2026-06-29T22:08:20.266Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWAPT6GD89R0QK5WS2A245QM duration_ms=33ns method=POST pid=1382 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 203.371416] gitaly[1382]: time="2026-06-29T22:08:20.278Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5319 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:19.637" grpc.time_ms=641.148 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.445785] gitaly[1382]: time="2026-06-29T22:08:20.353Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:20.351" grpc.time_ms=1.094 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.514766] gitaly[1382]: time="2026-06-29T22:08:20.421Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:20.420" grpc.time_ms=1.013 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.557864] gitaly[1382]: time="2026-06-29T22:08:20.465Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:20.462" grpc.time_ms=2.064 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.777484] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:20.683Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7feafdc30211c0e520a70ef8","created_at":"2026-06-29T22:08:20.631Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-406da90b660f493d","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:20.673Z","job_size_bytes":185,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7feafdc30211c0e520a70ef8: start","job_status":"start","queue_duration_s":0.010518,"scheduling_latency_s":0.010558} vm-test-run-nixbot-gitlab> gitlab # [ 203.804686] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:20.702Z","message":"changes: \"0000000000000000000000000000000000000000 1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 203.823214] gitaly[1382]: time="2026-06-29T22:08:20.730Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T22:08:30.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:20.724" grpc.time_ms=6.129 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.878738] gitaly[1382]: time="2026-06-29T22:08:20.785Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:30.056" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:20.777" grpc.time_ms=8.459 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.900375] gitaly[1382]: time="2026-06-29T22:08:20.794Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T22:08:30.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:20.786" grpc.time_ms=8.365 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.922532] gitaly[1382]: time="2026-06-29T22:08:20.820Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1566 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T22:08:30.074" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:20.795" grpc.time_ms=23.953 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.944150] gitaly[1382]: time="2026-06-29T22:08:20.829Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1569 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T22:08:30.028" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:20.808" grpc.time_ms=20.395 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.966337] gitaly[1382]: time="2026-06-29T22:08:20.868Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T22:08:30.138" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:20.859" grpc.time_ms=9.505 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.995617] gitaly[1382]: time="2026-06-29T22:08:20.879Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWAPT6GD89R0QK5WS2A245QM duration_ms=373ns method=POST pid=1382 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 204.001161] gitaly[1382]: time="2026-06-29T22:08:20.880Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4813 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:20.505" grpc.time_ms=374.832 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.018430] gitaly[1382]: time="2026-06-29T22:08:20.907Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3940 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:20.904" grpc.time_ms=2.918 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 204.034377] gitaly[1382]: time="2026-06-29T22:08:20.911Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM diskcache=4448a455-ea86-4435-9298-287ce0803eba grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:19.550" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.035098] gitlab-workhorse[1282]: time="2026-06-29T22:08:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # [ 204.036189] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:20 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 1442 vm-test-run-nixbot-gitlab> gitlab # [ 204.037402] setup-gitlab-start[3006]: To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 204.038274] setup-gitlab-start[3006]: * [new branch] master -> master vm-test-run-nixbot-gitlab> gitlab # [ 204.039237] gitaly[1382]: time="2026-06-29T22:08:20.911Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=280 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13757 command.oublock=32 command.real_time_ms=1347 command.spawn_token_fork_ms=0 command.system_time_ms=194 command.user_time_ms=86 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:19.550" grpc.time_ms=1360.267 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.045353] gitaly[1382]: time="2026-06-29T22:08:20.952Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:20.952" grpc.time_ms=0.466 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 204.095848] setup-gitlab-start[3006]: branch 'master' set up to track 'origin/master'. vm-test-run-nixbot-gitlab> gitlab # [ 204.099884] systemd[1]: Finished setup-gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 204.104653] systemd[1]: Starting nixbot CI service... vm-test-run-nixbot-gitlab> gitlab # [ 204.108113] gitaly[1382]: time="2026-06-29T22:08:21.015Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM diskcache=9a5e12b4-91a1-468c-84ed-f8746d190eb7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:08:30.158" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:20.878" pid=1382 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.110289] gitaly[1382]: time="2026-06-29T22:08:21.015Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=54 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4732 command.oublock=8 command.real_time_ms=126 command.spawn_token_fork_ms=1 command.system_time_ms=34 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:08:30.158" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:20.878" grpc.time_ms=136.246 pid=1382 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.126845] gitaly[1382]: time="2026-06-29T22:08:21.034Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T22:08:31.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:21.033" grpc.time_ms=0.177 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.141479] gitaly[1382]: time="2026-06-29T22:08:21.048Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=945 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T22:08:31.009" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:21.039" grpc.time_ms=8.872 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.209269] gitaly[1382]: time="2026-06-29T22:08:21.116Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-29T22:08:51.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:21.111" grpc.time_ms=4.655 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.232933] gitaly[1382]: time="2026-06-29T22:08:21.140Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T22:08:31.109" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:21.139" grpc.time_ms=0.232 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.259430] gitaly[1382]: time="2026-06-29T22:08:21.166Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T22:08:31.131" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T22:08:21.162" grpc.time_ms=4.249 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.298372] gitaly[1382]: time="2026-06-29T22:08:21.205Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:31.168" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:21.198" grpc.time_ms=6.187 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.309633] gitaly[1382]: time="2026-06-29T22:08:21.216Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:31.180" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:21.210" grpc.time_ms=6.444 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.331121] gitaly[1382]: time="2026-06-29T22:08:21.238Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T22:08:51.126" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:21.227" grpc.time_ms=11.16 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.618117] gitaly[1382]: time="2026-06-29T22:08:21.525Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T22:08:31.001" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:21.523" grpc.time_ms=1.754 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.656797] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:21.563Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7feafdc30211c0e520a70ef8","created_at":"2026-06-29T22:08:20.631Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-406da90b660f493d","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:20.673Z","job_size_bytes":185,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7feafdc30211c0e520a70ef8: done: 0.879399 sec","job_status":"done","queue_duration_s":0.010518,"scheduling_latency_s":0.010558,"gitaly_calls":14,"gitaly_duration_s":0.311462,"redis_calls":201,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.081397,"redis_read_bytes":922,"redis_write_bytes":19885,"redis_cache_calls":14,"redis_cache_duration_s":0.004539,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":32,"redis_feature_flag_duration_s":0.027508,"redis_feature_flag_read_bytes":540,"redis_feature_flag_write_bytes":4279,"redis_queues_calls":8,"redis_queues_duration_s":0.00151,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4936,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003342,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":92,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.025415,"redis_repository_cache_read_bytes":214,"redis_repository_cache_write_bytes":6392,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.019083,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":51,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":51,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":51,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.205,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.223,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.427055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002369575,"exclusive_lock_hold_duration_s":0.043377021,"duration_s":0.879399,"completed_at":"2026-06-29T22:08:21.563Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.083133,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 204.748471] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:21.654Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b050f65f5217ad8e11540ee3","created_at":"2026-06-29T22:08:20.823Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-2b8d8688a02b4268","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:20.830Z","job_size_bytes":46,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b050f65f5217ad8e11540ee3: start","job_status":"start","queue_duration_s":0.823787,"scheduling_latency_s":0.823863} vm-test-run-nixbot-gitlab> gitlab # [ 204.830941] gitaly[1382]: time="2026-06-29T22:08:21.738Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-30T04:08:21.735" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:21.735" grpc.time_ms=2.206 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.849372] gitaly[1382]: time="2026-06-29T22:08:21.752Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T22:08:51.644" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:21.745" grpc.time_ms=6.792 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.871350] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:21.777Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b050f65f5217ad8e11540ee3","created_at":"2026-06-29T22:08:20.823Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-2b8d8688a02b4268","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:20.830Z","job_size_bytes":46,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b050f65f5217ad8e11540ee3: done: 0.123392 sec","job_status":"done","queue_duration_s":0.823787,"scheduling_latency_s":0.823863,"gitaly_calls":2,"gitaly_duration_s":0.014035,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.012781,"redis_read_bytes":68,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002303,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000658,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002869,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.005433,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001518,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000403683,"duration_s":0.123392,"completed_at":"2026-06-29T22:08:21.777Z","load_balancing_strategy":"primary","db_duration_s":0.0187,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 204.876828] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:21.780Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"74dea50b05d38442ea80e4d6","created_at":"2026-06-29T22:08:20.848Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-2b8d8688a02b4268","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:20.853Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-74dea50b05d38442ea80e4d6: start","job_status":"start","queue_duration_s":0.926079,"scheduling_latency_s":0.926104} vm-test-run-nixbot-gitlab> gitlab # [ 204.961995] gitaly[1382]: time="2026-06-29T22:08:21.834Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM diskcache=451224d1-df6c-49f1-880c-f9e1a80382e8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T04:08:21.805" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:21.806" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.963353] gitaly[1382]: time="2026-06-29T22:08:21.834Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1282 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T04:08:21.805" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:21.806" grpc.time_ms=28.179 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.969175] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:21.876Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"74dea50b05d38442ea80e4d6","created_at":"2026-06-29T22:08:20.848Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-2b8d8688a02b4268","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:20.853Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-74dea50b05d38442ea80e4d6: done: 0.096103 sec","job_status":"done","queue_duration_s":0.926079,"scheduling_latency_s":0.926104,"gitaly_calls":1,"gitaly_duration_s":0.031141,"redis_calls":15,"redis_duration_s":0.0059759999999999995,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000877,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002031,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001237,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001831,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.071500149,"duration_s":0.096103,"completed_at":"2026-06-29T22:08:21.876Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016736,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.052220] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:21.877Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"ec5ffc2e7e0b535527b31616","created_at":"2026-06-29T22:08:21.028Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-2b8d8688a02b4268","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:21.030Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ec5ffc2e7e0b535527b31616: start","job_status":"start","queue_duration_s":0.847161,"scheduling_latency_s":0.847185} vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 206.14 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 205.053534] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:21.916Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"ec5ffc2e7e0b535527b31616","created_at":"2026-06-29T22:08:21.028Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-2b8d8688a02b4268","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:21.030Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ec5ffc2e7e0b535527b31616: done: 0.038281 sec","job_status":"done","queue_duration_s":0.847161,"scheduling_latency_s":0.847185,"gitaly_calls":1,"gitaly_duration_s":0.009513,"redis_calls":16,"redis_duration_s":0.0038929999999999998,"redis_read_bytes":52,"redis_write_bytes":2045,"redis_queues_calls":1,"redis_queues_duration_s":0.000282,"redis_queues_read_bytes":1,"redis_queues_write_bytes":976,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001187,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001647,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038281,"completed_at":"2026-06-29T22:08:21.915Z","load_balancing_strategy":"primary","db_duration_s":0.002429,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service vm-test-run-nixbot-gitlab> gitlab # [ 205.127358] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:21.917Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1766c212fe723f8cc00f0f21","created_at":"2026-06-29T22:08:21.483Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-2b8d8688a02b4268","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e14b1d5db948be12102899487f8b4f3b6d8d2bcd5b6683d1da4ae18283886b3b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:21.488Z","job_size_bytes":467,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1766c212fe723f8cc00f0f21: start","job_status":"start","queue_duration_s":0.428845,"scheduling_latency_s":0.42887} vm-test-run-nixbot-gitlab> gitlab # [ 205.197580] gitaly[1382]: time="2026-06-29T22:08:21.908Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T22:08:51.800" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:21.901" grpc.time_ms=7.12 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 205.198942] gitaly[1382]: time="2026-06-29T22:08:22.068Z" level=info msg="finished unary call with code OK" catfile.duration_ms=117 catfile.flush_count=1 catfile.flush_ms=87 catfile.read_object_count=1 catfile.read_object_ms=30 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:51.849" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:21.949" grpc.time_ms=118.183 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 205.200074] gitaly[1382]: time="2026-06-29T22:08:22.081Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T22:08:32.041" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:22.072" grpc.time_ms=9.466 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 205.342413] gitaly[1382]: time="2026-06-29T22:08:22.249Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:08:52.147" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:22.247" grpc.time_ms=1.04 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 205.374111] gitaly[1382]: time="2026-06-29T22:08:22.281Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=99 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T22:08:52.176" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:22.276" grpc.time_ms=4.528 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 205.568966] gitaly[1382]: time="2026-06-29T22:08:22.476Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:32.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:22.465" grpc.time_ms=10.049 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 205.624994] nixbot[3139]: {"ts": "2026-06-29T22:08:22.531Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} vm-test-run-nixbot-gitlab> gitlab # [ 205.657050] nixbot[3139]: {"ts": "2026-06-29T22:08:22.564Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} vm-test-run-nixbot-gitlab> gitlab # [ 205.678341] nixbot[3139]: {"ts": "2026-06-29T22:08:22.585Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 205.686720] nixbot[3139]: {"ts": "2026-06-29T22:08:22.589Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 205.687683] nixbot[3139]: {"ts": "2026-06-29T22:08:22.592Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 205.702396] nixbot[3139]: {"ts": "2026-06-29T22:08:22.609Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 205.713493] nixbot[3139]: {"ts": "2026-06-29T22:08:22.620Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 205.719758] nixbot[3139]: {"ts": "2026-06-29T22:08:22.626Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 205.729572] nixbot[3139]: {"ts": "2026-06-29T22:08:22.636Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} vm-test-run-nixbot-gitlab> gitlab # [ 205.733994] nixbot[3139]: {"ts": "2026-06-29T22:08:22.641Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 205.742561] nixbot[3139]: {"ts": "2026-06-29T22:08:22.649Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} vm-test-run-nixbot-gitlab> gitlab # [ 205.754904] nixbot[3139]: {"ts": "2026-06-29T22:08:22.662Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} vm-test-run-nixbot-gitlab> gitlab # [ 205.759920] nixbot[3139]: {"ts": "2026-06-29T22:08:22.666Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} vm-test-run-nixbot-gitlab> gitlab # [ 205.766936] nixbot[3139]: {"ts": "2026-06-29T22:08:22.674Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} vm-test-run-nixbot-gitlab> gitlab # [ 205.774904] nixbot[3139]: {"ts": "2026-06-29T22:08:22.682Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} vm-test-run-nixbot-gitlab> gitlab # [ 205.781370] nixbot[3139]: {"ts": "2026-06-29T22:08:22.688Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} vm-test-run-nixbot-gitlab> gitlab # [ 205.788230] nixbot[3139]: {"ts": "2026-06-29T22:08:22.695Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} vm-test-run-nixbot-gitlab> gitlab # [ 205.793677] nixbot[3139]: {"ts": "2026-06-29T22:08:22.700Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 205.802994] nixbot[3139]: {"ts": "2026-06-29T22:08:22.710Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} vm-test-run-nixbot-gitlab> gitlab # [ 206.120848] nixbot[3139]: INFO: Started server process [3139] vm-test-run-nixbot-gitlab> gitlab # [ 206.124229] nixbot[3139]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) vm-test-run-nixbot-gitlab> gitlab # [ 206.138359] nixbot[3139]: INFO: ::ffff:127.0.0.1:53282 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 206.143568] systemd[1]: Started nixbot CI service. vm-test-run-nixbot-gitlab> gitlab # [ 206.146200] systemd[1]: Reached target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 206.148157] systemd[1]: Startup finished in 1.545s (kernel) + 4.390s (initrd) + 3min 20.212s (userspace) = 3min 26.147s. vm-test-run-nixbot-gitlab> gitlab # [ 206.217223] gitaly[1382]: time="2026-06-29T22:08:23.124Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T22:08:53.008" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:23.108" grpc.time_ms=15.469 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 206.264641] gitaly[1382]: time="2026-06-29T22:08:23.171Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPT6GD89R0QK5WS2A245QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T22:08:53.061" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:23.161" grpc.time_ms=9.834 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 206.301972] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T22:08:23.208Z","class":"CreatePipelineWorker","correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 206.319572] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:23.226Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1766c212fe723f8cc00f0f21","created_at":"2026-06-29T22:08:21.483Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-2b8d8688a02b4268","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e14b1d5db948be12102899487f8b4f3b6d8d2bcd5b6683d1da4ae18283886b3b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:21.488Z","job_size_bytes":467,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1766c212fe723f8cc00f0f21: done: 1.308989 sec","job_status":"done","queue_duration_s":0.428845,"scheduling_latency_s":0.42887,"gitaly_calls":7,"gitaly_duration_s":0.369812,"redis_calls":53,"redis_duration_s":0.027264,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.001781,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.007281,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004394,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.804903,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000357866,"duration_s":1.308989,"completed_at":"2026-06-29T22:08:23.226Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.068604,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 206.327160] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:23.229Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6c98b9edf75771507dff9d61","created_at":"2026-06-29T22:08:21.762Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-5dffdf3aa24f4a44","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:21.767Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6c98b9edf75771507dff9d61: start","job_status":"start","queue_duration_s":1.461673,"scheduling_latency_s":1.461696} vm-test-run-nixbot-gitlab> gitlab # [ 206.422151] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:23.266Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6c98b9edf75771507dff9d61","created_at":"2026-06-29T22:08:21.762Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-5dffdf3aa24f4a44","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:21.767Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6c98b9edf75771507dff9d61: done: 0.037994 sec","job_status":"done","queue_duration_s":1.461673,"scheduling_latency_s":1.461696,"redis_calls":19,"redis_duration_s":0.004493,"redis_read_bytes":21,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000871,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.00056,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002159,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00660922,"duration_s":0.037994,"completed_at":"2026-06-29T22:08:23.266Z","load_balancing_strategy":"primary","db_duration_s":0.006445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 206.424397] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:23.268Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"3770dfa3c55762151e4474cd","created_at":"2026-06-29T22:08:23.255Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-3ed48f21b2d0452c","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:23.259Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-3770dfa3c55762151e4474cd: start","job_status":"start","queue_duration_s":0.008635,"scheduling_latency_s":0.008656} vm-test-run-nixbot-gitlab> gitlab # [ 206.525190] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:23.431Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"3770dfa3c55762151e4474cd","created_at":"2026-06-29T22:08:23.255Z","trace_propagation_headers":{"sentry-trace":"685a022f3f6c47408ba387c71f9586fc-3ed48f21b2d0452c","baggage":"sentry-trace_id=685a022f3f6c47408ba387c71f9586fc,sentry-release=Unknown"},"correlation_id":"01KWAPT6GD89R0QK5WS2A245QM","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:23.259Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-3770dfa3c55762151e4474cd: done: 0.162857 sec","job_status":"done","queue_duration_s":0.008635,"scheduling_latency_s":0.008656,"redis_calls":11,"redis_duration_s":0.00397,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001413,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002107,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00045,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.139388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000403404,"duration_s":0.162857,"completed_at":"2026-06-29T22:08:23.431Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 1.32 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health vm-test-run-nixbot-gitlab> gitlab # [ 206.644044] nixbot[3139]: INFO: ::ffff:127.0.0.1:53298 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds) vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 206.733762] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 61 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 207.462859] gitaly[1382]: time="2026-06-29T22:08:24.369Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTA0333BF7DNTFFHDAP5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T22:08:54.349" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:24.349" grpc.time_ms=18.92 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 207.828583] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 43 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 207.865680] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:24 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1728 vm-test-run-nixbot-gitlab> gitlab # [ 207.867505] nixbot[3139]: {"ts": "2026-06-29T22:08:24.774Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 207.898546] nixbot[3139]: {"ts": "2026-06-29T22:08:24.805Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} vm-test-run-nixbot-gitlab> gitlab # [ 207.976161] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 59 vm-test-run-nixbot-gitlab> gitlab # [ 207.979142] nixbot[3139]: {"ts": "2026-06-29T22:08:24.884Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 207.982421] nixbot[3139]: {"ts": "2026-06-29T22:08:24.887Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} vm-test-run-nixbot-gitlab> gitlab # [ 208.152636] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 149 vm-test-run-nixbot-gitlab> gitlab # [ 208.154142] nixbot[3139]: {"ts": "2026-06-29T22:08:25.060Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 208.315145] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 151 vm-test-run-nixbot-gitlab> gitlab # [ 208.315467] nixbot[3139]: {"ts": "2026-06-29T22:08:25.221Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 208.412437] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 76 vm-test-run-nixbot-gitlab> gitlab # [ 208.413831] nixbot[3139]: {"ts": "2026-06-29T22:08:25.320Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 208.516285] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 98 vm-test-run-nixbot-gitlab> gitlab # [ 208.526470] nixbot[3139]: {"ts": "2026-06-29T22:08:25.433Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 208.605438] gitaly[1382]: time="2026-06-29T22:08:25.510Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCBEGVGWXKR5Q6AE0D04 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-29T22:08:55.094" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:25.495" grpc.time_ms=14.83 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 208.626143] gitaly[1382]: time="2026-06-29T22:08:25.533Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCBEGVGWXKR5Q6AE0D04 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:55.131" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:25.532" grpc.time_ms=0.891 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 208.643067] gitaly[1382]: time="2026-06-29T22:08:25.550Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCBEGVGWXKR5Q6AE0D04 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T22:08:35.009" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:25.540" grpc.time_ms=9.743 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 208.699455] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 151 vm-test-run-nixbot-gitlab> gitlab # [ 208.701183] nixbot[3139]: {"ts": "2026-06-29T22:08:25.607Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 208.764870] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 61 vm-test-run-nixbot-gitlab> gitlab # [ 208.766420] nixbot[3139]: {"ts": "2026-06-29T22:08:25.672Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 208.787407] nixbot[3139]: {"ts": "2026-06-29T22:08:25.694Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 208.922300] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 48 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-29T22:08:25.044Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 2.28 seconds) vm-test-run-nixbot-gitlab> subtest: push delivers the webhook vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master vm-test-run-nixbot-gitlab> gitlab # [ 208.934084] gitaly[1382]: time="2026-06-29T22:08:25.841Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=10 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCMKQCM7JEQJ835T7DDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:25.823" grpc.time_ms=17.033 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 208.936355] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 94 vm-test-run-nixbot-gitlab> gitlab # [ 209.022942] gitaly[1382]: time="2026-06-29T22:08:25.928Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCQNHG0R44WYNG2M774K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:25.915" pid=1382 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.025370] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 36 vm-test-run-nixbot-gitlab> gitlab # [ 209.027958] gitaly[1382]: time="2026-06-29T22:08:25.928Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCQNHG0R44WYNG2M774K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:25.915" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.029366] gitlab-workhorse[1282]: time="2026-06-29T22:08:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=149 vm-test-run-nixbot-gitlab> gitlab # [ 209.030220] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 88 vm-test-run-nixbot-gitlab> gitlab # [ 209.031166] gitaly[1382]: time="2026-06-29T22:08:25.932Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCQNHG0R44WYNG2M774K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:25.915" grpc.time_ms=16.469 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.109753] gitaly[1382]: time="2026-06-29T22:08:26.017Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCTBATFHZ8X139ETED09 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.008" grpc.time_ms=8.603 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.132059] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:26 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 200 "" "git/2.54.0" 87 vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done. vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done. vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) vm-test-run-nixbot-gitlab> gitlab # [ 209.159527] gitaly[1382]: time="2026-06-29T22:08:26.064Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCWMBRDJ66CM7KBGA3DP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.047" pid=1382 remote_ip="2001:db8:1::1" request_sha=b4f465bf72278947ee64342c5a2a2a61ece65db5 response_bytes=373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.175929] gitlab-workhorse[1282]: time="2026-06-29T22:08:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=373 vm-test-run-nixbot-gitlab> gitlab # [ 209.179341] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 373 "" "git/2.54.0" 60 vm-test-run-nixbot-gitlab> gitlab # [ 209.182926] gitaly[1382]: time="2026-06-29T22:08:26.064Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1226 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCWMBRDJ66CM7KBGA3DP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.047" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.205060] gitaly[1382]: time="2026-06-29T22:08:26.066Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1226 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCWMBRDJ66CM7KBGA3DP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.047" grpc.time_ms=17.643 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.329117] gitaly[1382]: time="2026-06-29T22:08:26.235Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=8 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTD1843PQ0JDHKDSWZ5BT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=196 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.204" grpc.time_ms=30.575 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.351547] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 105 vm-test-run-nixbot-gitlab> gitlab # [ 209.394137] gitaly[1382]: time="2026-06-29T22:08:26.300Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T22:08:36.067" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:26.288" grpc.time_ms=12.537 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.424096] gitaly[1382]: time="2026-06-29T22:08:26.331Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=7 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T22:08:56.017" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T22:08:26.318" grpc.time_ms=13.185 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.440753] gitaly[1382]: time="2026-06-29T22:08:26.347Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R duration_ms=117ns method=POST pid=1382 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 209.470380] gitaly[1382]: time="2026-06-29T22:08:26.377Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R duration_ms=28ns method=POST pid=1382 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 209.475931] gitaly[1382]: time="2026-06-29T22:08:26.383Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5319 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:26.229" grpc.time_ms=150.554 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.496621] gitaly[1382]: time="2026-06-29T22:08:26.403Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTD4MY57EPV2H5DCD2PW8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.366" pid=1382 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.497505] gitaly[1382]: time="2026-06-29T22:08:26.404Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=7 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTD4MY57EPV2H5DCD2PW8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.366" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.520362] gitaly[1382]: time="2026-06-29T22:08:26.427Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:26.425" grpc.time_ms=1.248 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.553829] gitaly[1382]: time="2026-06-29T22:08:26.461Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=7 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTD4MY57EPV2H5DCD2PW8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.366" grpc.time_ms=94.548 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 209.578525] gitlab-workhorse[1282]: time="2026-06-29T22:08:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 209.578752] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 224 vm-test-run-nixbot-gitlab> gitlab # [ 209.594711] gitaly[1382]: time="2026-06-29T22:08:26.501Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:26.500" grpc.time_ms=1.047 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.635166] gitaly[1382]: time="2026-06-29T22:08:26.541Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:26.541" grpc.time_ms=0.34 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.673534] gitaly[1382]: time="2026-06-29T22:08:26.580Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:56.278" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:26.579" grpc.time_ms=0.866 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.718077] gitaly[1382]: time="2026-06-29T22:08:26.625Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T22:08:56.316" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:26.617" grpc.time_ms=7.878 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.737329] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:26.643Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f c0de400263dc825e5f844f505cbeaf8de2ec47cd refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"35e00cb43b203260f3a8b516","created_at":"2026-06-29T22:08:26.638Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-a59b93580937497f","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.641Z","job_size_bytes":185,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-35e00cb43b203260f3a8b516: start","job_status":"start","queue_duration_s":0.001647,"scheduling_latency_s":0.001688} vm-test-run-nixbot-gitlab> gitlab # [ 209.749959] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:26.656Z","message":"changes: \"1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f c0de400263dc825e5f844f505cbeaf8de2ec47cd refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 209.755783] gitaly[1382]: time="2026-06-29T22:08:26.662Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R duration_ms=83ns method=POST pid=1382 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 209.759738] gitaly[1382]: time="2026-06-29T22:08:26.666Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4813 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:26.578" grpc.time_ms=84.927 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.783119] gitaly[1382]: time="2026-06-29T22:08:26.689Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:36.222" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:26.682" grpc.time_ms=6.46 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.799086] gitaly[1382]: time="2026-06-29T22:08:26.706Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T22:08:36.240" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:26.701" grpc.time_ms=5.113 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.834122] gitaly[1382]: time="2026-06-29T22:08:26.740Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T22:08:36.271" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:26.731" grpc.time_ms=9.136 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.848664] gitaly[1382]: time="2026-06-29T22:08:26.755Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-29T22:08:56.645" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:26.746" grpc.time_ms=9.645 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.865394] gitaly[1382]: time="2026-06-29T22:08:26.772Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T22:08:36.247" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T22:08:26.767" grpc.time_ms=5.176 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.879416] gitaly[1382]: time="2026-06-29T22:08:26.782Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=146 command.spawn_token_fork_ms=140 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:36.084" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:26.634" grpc.time_ms=147.632 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.880359] gitaly[1382]: time="2026-06-29T22:08:26.786Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:36.259" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:26.779" grpc.time_ms=6.599 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.881191] gitaly[1382]: time="2026-06-29T22:08:26.786Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R diskcache=7d3f9fe5-9c77-4c24-9bef-065e5ed3ca64 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.154" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.955974] gitaly[1382]: time="2026-06-29T22:08:26.786Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=263 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=12996 command.oublock=32 command.real_time_ms=617 command.spawn_token_fork_ms=3 command.system_time_ms=210 command.user_time_ms=53 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:26.154" grpc.time_ms=632.018 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.959530] gitlab-workhorse[1282]: time="2026-06-29T22:08:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # [ 209.960389] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:26 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 703 vm-test-run-nixbot-gitlab> gitlab # [ 209.963180] gitaly[1382]: time="2026-06-29T22:08:26.842Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T22:08:36.001" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:26.842" grpc.time_ms=0.322 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # 1ab2c6a..c0de400 master -> master vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.10 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master vm-test-run-nixbot-gitlab> gitlab # [ 210.017669] gitaly[1382]: time="2026-06-29T22:08:26.924Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=618 command.oublock=0 command.real_time_ms=64 command.spawn_token_fork_ms=58 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T22:08:36.017" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:26.857" grpc.time_ms=66.211 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd' vm-test-run-nixbot-gitlab> gitlab # [ 210.057380] gitaly[1382]: time="2026-06-29T22:08:26.964Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:26.963" grpc.time_ms=0.737 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 210.080945] nixbot[3139]: INFO: ::ffff:127.0.0.1:53300 - "GET /api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd', in 0.04 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 210.101210] gitaly[1382]: time="2026-06-29T22:08:27.008Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:27.006" grpc.time_ms=1.733 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 210.124720] gitaly[1382]: time="2026-06-29T22:08:27.031Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS diskcache=1b3c789d-90dc-429d-aad0-edd495c6acee grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:08:36.086" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:26.927" pid=1382 ref=refs/keep-around/1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f remote_ip="2001:db8:1::1" revision=1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.132539] gitaly[1382]: time="2026-06-29T22:08:27.039Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T22:08:37.006" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:27.036" grpc.time_ms=3.341 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.163963] gitaly[1382]: time="2026-06-29T22:08:27.071Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=73 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5143 command.oublock=8 command.real_time_ms=96 command.spawn_token_fork_ms=1 command.system_time_ms=53 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:08:36.086" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:26.927" grpc.time_ms=104.808 pid=1382 ref=refs/keep-around/1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f remote_ip="2001:db8:1::1" revision=1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.169261] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:27.076Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f c0de400263dc825e5f844f505cbeaf8de2ec47cd refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"35e00cb43b203260f3a8b516","created_at":"2026-06-29T22:08:26.638Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-a59b93580937497f","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.641Z","job_size_bytes":185,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-35e00cb43b203260f3a8b516: done: 0.432701 sec","job_status":"done","queue_duration_s":0.001647,"scheduling_latency_s":0.001688,"gitaly_calls":7,"gitaly_duration_s":0.078213,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.056458,"redis_read_bytes":350,"redis_write_bytes":13068,"redis_cache_calls":9,"redis_cache_duration_s":0.00318,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005843,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.003663,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6768,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.005134,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.016947,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.021691,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.028,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.028,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.182399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.005208483,"exclusive_lock_hold_duration_s":0.023029463,"duration_s":0.432701,"completed_at":"2026-06-29T22:08:27.076Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.093539,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 210.241234] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:27.081Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"2809749c6f43c0615eb49718","created_at":"2026-06-29T22:08:26.718Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-d2b82b3539a740dc","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.720Z","job_size_bytes":46,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2809749c6f43c0615eb49718: start","job_status":"start","queue_duration_s":0.361034,"scheduling_latency_s":0.361082} vm-test-run-nixbot-gitlab> gitlab # [ 210.242745] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:27.118Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"2809749c6f43c0615eb49718","created_at":"2026-06-29T22:08:26.718Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-d2b82b3539a740dc","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.720Z","job_size_bytes":46,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2809749c6f43c0615eb49718: done: 0.0366 sec","job_status":"done","queue_duration_s":0.361034,"scheduling_latency_s":0.361082,"redis_calls":9,"redis_duration_s":0.008359,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.001154,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00303,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0366,"completed_at":"2026-06-29T22:08:27.118Z","load_balancing_strategy":"primary","db_duration_s":0.004662,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 210.346178] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:27.122Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"4879b6f891429ae2a9517d2d","created_at":"2026-06-29T22:08:26.726Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-d2b82b3539a740dc","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.727Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-4879b6f891429ae2a9517d2d: start","job_status":"start","queue_duration_s":0.395133,"scheduling_latency_s":0.395178} vm-test-run-nixbot-gitlab> gitlab # [ 210.396141] gitaly[1382]: time="2026-06-29T22:08:27.303Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:57.099" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:27.299" grpc.time_ms=3.323 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.408815] gitaly[1382]: time="2026-06-29T22:08:27.316Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R diskcache=0ace3127-7757-483d-a472-963c99e0b71f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T04:08:27.223" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:27.224" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.411193] gitaly[1382]: time="2026-06-29T22:08:27.318Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1648 command.oublock=0 command.real_time_ms=90 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T04:08:27.223" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:27.224" grpc.time_ms=93.351 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.476496] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:27.383Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"4879b6f891429ae2a9517d2d","created_at":"2026-06-29T22:08:26.726Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-d2b82b3539a740dc","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.727Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-4879b6f891429ae2a9517d2d: done: 0.260476 sec","job_status":"done","queue_duration_s":0.395133,"scheduling_latency_s":0.395178,"gitaly_calls":1,"gitaly_duration_s":0.112497,"redis_calls":12,"redis_duration_s":0.016080999999999998,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006901,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.004227,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004953,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.02,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.02,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.21885519,"duration_s":0.260476,"completed_at":"2026-06-29T22:08:27.383Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 210.484103] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:27.391Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"29ab5a86a481dd4f83bc8416","created_at":"2026-06-29T22:08:26.844Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-d2b82b3539a740dc","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d76c01218aa9cc4405bba3195bd4b516af4d22ceadf4446863e36d19c4c9a825","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.847Z","job_size_bytes":467,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-29ab5a86a481dd4f83bc8416: start","job_status":"start","queue_duration_s":0.543922,"scheduling_latency_s":0.543959} vm-test-run-nixbot-gitlab> gitlab # [ 210.527384] gitaly[1382]: time="2026-06-29T22:08:27.434Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:57.331" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:27.432" grpc.time_ms=0.662 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.542360] gitaly[1382]: time="2026-06-29T22:08:27.449Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T22:08:37.006" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:27.437" grpc.time_ms=11.726 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.586188] gitaly[1382]: time="2026-06-29T22:08:27.493Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:08:57.391" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:27.492" grpc.time_ms=0.997 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.594084] gitaly[1382]: time="2026-06-29T22:08:27.501Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=197 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T22:08:57.398" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:27.498" grpc.time_ms=2.833 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.652672] gitaly[1382]: time="2026-06-29T22:08:27.558Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:37.089" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:27.550" grpc.time_ms=8.622 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.723499] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 1104 vm-test-run-nixbot-gitlab> gitlab # [ 210.724616] nixbot[3139]: {"ts": "2026-06-29T22:08:27.631Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 210.731938] nixbot[3139]: {"ts": "2026-06-29T22:08:27.639Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-1ab2c6ae3a34-434fee3f", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 210.813854] systemd[1]: Started Nix Daemon. vm-test-run-nixbot-gitlab> gitlab # [ 210.863923] nix-daemon[3417]: accepted connection from pid 3416, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 210.868769] nix-daemon[3417]: accepted connection from pid 3415, user nixbot vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd' vm-test-run-nixbot-gitlab> gitlab # [ 211.133789] gitaly[1382]: time="2026-06-29T22:08:28.040Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=4 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T22:08:58.011" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:28.012" grpc.time_ms=27.984 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 211.139671] nixbot[3139]: INFO: ::ffff:127.0.0.1:53302 - "GET /api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd', in 0.09 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 211.198164] gitaly[1382]: time="2026-06-29T22:08:28.104Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTCZ4J6EV5S2VAXBBD45R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T22:08:58.090" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:28.090" grpc.time_ms=14.077 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 211.225742] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T22:08:28.132Z","class":"CreatePipelineWorker","correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f","after":"c0de400263dc825e5f844f505cbeaf8de2ec47cd","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"c0de400263dc825e5f844f505cbeaf8de2ec47cd","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 211.245274] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.152Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"29ab5a86a481dd4f83bc8416","created_at":"2026-06-29T22:08:26.844Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-d2b82b3539a740dc","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d76c01218aa9cc4405bba3195bd4b516af4d22ceadf4446863e36d19c4c9a825","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.847Z","job_size_bytes":467,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-29ab5a86a481dd4f83bc8416: done: 0.761061 sec","job_status":"done","queue_duration_s":0.543922,"scheduling_latency_s":0.543959,"gitaly_calls":7,"gitaly_duration_s":0.175448,"redis_calls":33,"redis_duration_s":0.019544,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001351,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005927,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.008883,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003383,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.458469,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.761061,"completed_at":"2026-06-29T22:08:28.152Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0582,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 211.282881] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.158Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"e027a564-c048-453c-b14f-24c236312e3a\"}"],"class":"WebHookWorker","jid":"7152ee16e1a946e3a5a4fed1","created_at":"2026-06-29T22:08:26.978Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-d2b82b3539a740dc","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4b63e8b76190128def452d2b2130d85985455d9cca35d461a50b45ef77648f3","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.997Z","job_size_bytes":1749,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-7152ee16e1a946e3a5a4fed1: start","job_status":"start","queue_duration_s":1.160666,"scheduling_latency_s":1.160701} vm-test-run-nixbot-gitlab> gitlab # [ 211.503939] nixbot[3139]: INFO: ::ffff:127.0.0.1:53316 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted vm-test-run-nixbot-gitlab> gitlab # [ 211.566542] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.473Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"e027a564-c048-453c-b14f-24c236312e3a\"}"],"class":"WebHookWorker","jid":"7152ee16e1a946e3a5a4fed1","created_at":"2026-06-29T22:08:26.978Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-d2b82b3539a740dc","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4b63e8b76190128def452d2b2130d85985455d9cca35d461a50b45ef77648f3","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:26.997Z","job_size_bytes":1749,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-7152ee16e1a946e3a5a4fed1: done: 0.314691 sec","job_status":"done","queue_duration_s":1.160666,"scheduling_latency_s":1.160701,"redis_calls":19,"redis_duration_s":0.033194,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.011145,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.001063,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.015733,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005253,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.076,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.076,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.02322669499997687,"cpu_s":0.11161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000860165,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.314691,"completed_at":"2026-06-29T22:08:28.472Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 211.575169] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.482Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"4816a1fa84c5909a6ee29eea","created_at":"2026-06-29T22:08:27.306Z","trace_propagation_headers":{"sentry-trace":"5f28f8180ee14bd79e4c3a73a23bad35-0908097e545d4f8d","baggage":"sentry-trace_id=5f28f8180ee14bd79e4c3a73a23bad35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTDCXFXJ3QS3HZ3SB0TMS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:27.312Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-4816a1fa84c5909a6ee29eea: start","job_status":"start","queue_duration_s":1.169822,"scheduling_latency_s":1.16986} vm-test-run-nixbot-gitlab> gitlab # [ 211.721519] gitaly[1382]: time="2026-06-29T22:08:28.627Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTF9CF25CJ2XAD1E4E2FY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:28.617" grpc.time_ms=9.401 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 211.723630] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 167 vm-test-run-nixbot-gitlab> gitlab # [ 211.774062] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.680Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"4816a1fa84c5909a6ee29eea","created_at":"2026-06-29T22:08:27.306Z","trace_propagation_headers":{"sentry-trace":"5f28f8180ee14bd79e4c3a73a23bad35-0908097e545d4f8d","baggage":"sentry-trace_id=5f28f8180ee14bd79e4c3a73a23bad35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTDCXFXJ3QS3HZ3SB0TMS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:27.312Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-4816a1fa84c5909a6ee29eea: done: 0.198774 sec","job_status":"done","queue_duration_s":1.169822,"scheduling_latency_s":1.16986,"redis_calls":18,"redis_duration_s":0.049618,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.047633,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001543,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.072,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000414019,"duration_s":0.198774,"completed_at":"2026-06-29T22:08:28.680Z","load_balancing_strategy":"primary","db_duration_s":0.089157,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 211.779060] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.686Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a37fc7061f8382199da65083","created_at":"2026-06-29T22:08:27.312Z","trace_propagation_headers":{"sentry-trace":"5f28f8180ee14bd79e4c3a73a23bad35-0908097e545d4f8d","baggage":"sentry-trace_id=5f28f8180ee14bd79e4c3a73a23bad35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTDCXFXJ3QS3HZ3SB0TMS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:27.322Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a37fc7061f8382199da65083: start","job_status":"start","queue_duration_s":1.363572,"scheduling_latency_s":1.363611} vm-test-run-nixbot-gitlab> gitlab # [ 211.806790] gitaly[1382]: time="2026-06-29T22:08:28.714Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTFEPMGKPH5SNZ8QXTW4N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:28.698" pid=1382 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 211.878494] gitaly[1382]: time="2026-06-29T22:08:28.714Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTFEPMGKPH5SNZ8QXTW4N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:28.698" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 211.879810] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.760Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a37fc7061f8382199da65083","created_at":"2026-06-29T22:08:27.312Z","trace_propagation_headers":{"sentry-trace":"5f28f8180ee14bd79e4c3a73a23bad35-0908097e545d4f8d","baggage":"sentry-trace_id=5f28f8180ee14bd79e4c3a73a23bad35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTDCXFXJ3QS3HZ3SB0TMS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:27.322Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a37fc7061f8382199da65083: done: 0.074806 sec","job_status":"done","queue_duration_s":1.363572,"scheduling_latency_s":1.363611,"redis_calls":15,"redis_duration_s":0.009416,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00628,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001606,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000417651,"duration_s":0.074806,"completed_at":"2026-06-29T22:08:28.760Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015668,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 211.972958] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.762Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"964de9ab65eae34f085646a4","created_at":"2026-06-29T22:08:27.613Z","trace_propagation_headers":{"sentry-trace":"5f28f8180ee14bd79e4c3a73a23bad35-0908097e545d4f8d","baggage":"sentry-trace_id=5f28f8180ee14bd79e4c3a73a23bad35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTDCXFXJ3QS3HZ3SB0TMS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:27.618Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-964de9ab65eae34f085646a4: start","job_status":"start","queue_duration_s":1.143338,"scheduling_latency_s":1.14336} vm-test-run-nixbot-gitlab> gitlab # [ 211.974947] gitlab-workhorse[1282]: time="2026-06-29T22:08:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 211.977073] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 85 vm-test-run-nixbot-gitlab> gitlab # [ 211.979054] gitaly[1382]: time="2026-06-29T22:08:28.714Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTFEPMGKPH5SNZ8QXTW4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:28.698" grpc.time_ms=16.193 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 211.980188] gitaly[1382]: time="2026-06-29T22:08:28.835Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:08:58.834" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:28.834" grpc.time_ms=0.997 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.057374] gitaly[1382]: time="2026-06-29T22:08:28.907Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:58.005" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:28.906" grpc.time_ms=1.178 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.059187] gitaly[1382]: time="2026-06-29T22:08:28.942Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTDCXFXJ3QS3HZ3SB0TMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:58.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:28.941" grpc.time_ms=0.588 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd' vm-test-run-nixbot-gitlab> gitlab # [ 212.060396] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.967Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"964de9ab65eae34f085646a4","created_at":"2026-06-29T22:08:27.613Z","trace_propagation_headers":{"sentry-trace":"5f28f8180ee14bd79e4c3a73a23bad35-0908097e545d4f8d","baggage":"sentry-trace_id=5f28f8180ee14bd79e4c3a73a23bad35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTDCXFXJ3QS3HZ3SB0TMS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:27.618Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-964de9ab65eae34f085646a4: done: 0.205056 sec","job_status":"done","queue_duration_s":1.143338,"scheduling_latency_s":1.14336,"gitaly_calls":3,"gitaly_duration_s":0.018641,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.026328,"redis_read_bytes":208,"redis_write_bytes":6802,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001148,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003198,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001803,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000298,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.014526,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00057,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.00165,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001987,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098184,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.16512489800000002,"duration_s":0.205056,"completed_at":"2026-06-29T22:08:28.967Z","load_balancing_strategy":"primary","db_duration_s":0.06514,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd', in 0.04 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 212.145329] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:28.970Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","7152ee16e1a946e3a5a4fed1"],"class":"WebHooks::LogExecutionWorker","jid":"1abaab0e598c64374b063598","created_at":"2026-06-29T22:08:28.426Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-0a6c5f2fa0fc4c16","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:41b80e967a957c445e4600a58ebaed463d0c76d244ab6dfe2969428c8c1678e7","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:28.451Z","job_size_bytes":2507,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-1abaab0e598c64374b063598: start","job_status":"start","queue_duration_s":0.519541,"scheduling_latency_s":0.519566} vm-test-run-nixbot-gitlab> gitlab # [ 212.216205] nixbot[3139]: INFO: ::ffff:127.0.0.1:53324 - "GET /api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 212.220952] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:29.127Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","7152ee16e1a946e3a5a4fed1"],"class":"WebHooks::LogExecutionWorker","jid":"1abaab0e598c64374b063598","created_at":"2026-06-29T22:08:28.426Z","trace_propagation_headers":{"sentry-trace":"b28d6d2bc0e8493696f620b098f2d170-0a6c5f2fa0fc4c16","baggage":"sentry-trace_id=b28d6d2bc0e8493696f620b098f2d170,sentry-release=Unknown"},"correlation_id":"01KWAPTCZ4J6EV5S2VAXBBD45R","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:41b80e967a957c445e4600a58ebaed463d0c76d244ab6dfe2969428c8c1678e7","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:28.451Z","job_size_bytes":2507,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-1abaab0e598c64374b063598: done: 0.157067 sec","job_status":"done","queue_duration_s":0.519541,"scheduling_latency_s":0.519566,"redis_calls":10,"redis_duration_s":0.005223,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001815,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002934,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.029,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.029,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000424076,"duration_s":0.157067,"completed_at":"2026-06-29T22:08:29.127Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028189,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 212.294299] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:29.129Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4adb4e6fcc352ce4fbffa1f2","created_at":"2026-06-29T22:08:28.819Z","trace_propagation_headers":{"sentry-trace":"5f28f8180ee14bd79e4c3a73a23bad35-e8712dd1633b4991","baggage":"sentry-trace_id=5f28f8180ee14bd79e4c3a73a23bad35,sentry-release=Unknown"},"correlation_id":"01KWAPTDCXFXJ3QS3HZ3SB0TMS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:28.824Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4adb4e6fcc352ce4fbffa1f2: start","job_status":"start","queue_duration_s":0.305551,"scheduling_latency_s":0.305575} vm-test-run-nixbot-gitlab> gitlab # [ 212.331457] gitaly[1382]: time="2026-06-29T22:08:29.238Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTFHF90JMN1T311MX5CYQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:28.915" pid=1382 remote_ip="2001:db8:1::1" request_sha=628ff87e1e83e01b9ddf3e45ecb63d0bcd1379df response_bytes=384 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 212.334574] gitaly[1382]: time="2026-06-29T22:08:29.239Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1207 command.oublock=0 command.real_time_ms=318 command.spawn_token_fork_ms=193 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTFHF90JMN1T311MX5CYQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:28.915" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 212.336383] gitlab-workhorse[1282]: time="2026-06-29T22:08:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=384 vm-test-run-nixbot-gitlab> gitlab # [ 212.337511] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 384 "" "git/2.54.0" 522 vm-test-run-nixbot-gitlab> gitlab # [ 212.338711] gitaly[1382]: time="2026-06-29T22:08:29.240Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1207 command.oublock=0 command.real_time_ms=318 command.spawn_token_fork_ms=193 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTFHF90JMN1T311MX5CYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:28.915" grpc.time_ms=324.543 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 212.422861] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:29.329Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4adb4e6fcc352ce4fbffa1f2","created_at":"2026-06-29T22:08:28.819Z","trace_propagation_headers":{"sentry-trace":"5f28f8180ee14bd79e4c3a73a23bad35-e8712dd1633b4991","baggage":"sentry-trace_id=5f28f8180ee14bd79e4c3a73a23bad35,sentry-release=Unknown"},"correlation_id":"01KWAPTDCXFXJ3QS3HZ3SB0TMS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:28.824Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4adb4e6fcc352ce4fbffa1f2: done: 0.200079 sec","job_status":"done","queue_duration_s":0.305551,"scheduling_latency_s":0.305575,"redis_calls":10,"redis_duration_s":0.010715,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007332,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.137,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.136,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001529803,"duration_s":0.200079,"completed_at":"2026-06-29T22:08:29.329Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.137371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 212.430439] nixbot[3139]: {"ts": "2026-06-29T22:08:29.337Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"} vm-test-run-nixbot-gitlab> gitlab # [ 212.441851] nix-daemon[3422]: unexpected Nix daemon error: error: interrupted by the user vm-test-run-nixbot-gitlab> gitlab # [ 212.551761] gitaly[1382]: time="2026-06-29T22:08:29.458Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:59.457" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:29.457" grpc.time_ms=0.544 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.565247] gitaly[1382]: time="2026-06-29T22:08:29.472Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:59.171" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:29.471" grpc.time_ms=0.481 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.601752] gitaly[1382]: time="2026-06-29T22:08:29.508Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:39.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:29.502" grpc.time_ms=6.477 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.629910] gitaly[1382]: time="2026-06-29T22:08:29.537Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T22:08:59.526" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:29.527" grpc.time_ms=9.329 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.651110] gitaly[1382]: time="2026-06-29T22:08:29.558Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:39.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:29.552" grpc.time_ms=5.968 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.695780] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 179 vm-test-run-nixbot-gitlab> gitlab # [ 212.696595] nixbot[3139]: {"ts": "2026-06-29T22:08:29.602Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 212.715075] gitaly[1382]: time="2026-06-29T22:08:29.622Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T22:08:39.071" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:29.621" grpc.time_ms=0.24 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.733240] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:29.637Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"075b19a886b57b942bebd9d9","created_at":"2026-06-29T22:08:29.537Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-cfb71e52ac514b86","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.585Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-075b19a886b57b942bebd9d9: start","job_status":"start","queue_duration_s":0.052527,"scheduling_latency_s":0.05257} vm-test-run-nixbot-gitlab> gitlab # [ 212.745507] gitaly[1382]: time="2026-06-29T22:08:29.652Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T22:08:39.094" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:29.645" grpc.time_ms=7.101 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.791770] gitaly[1382]: time="2026-06-29T22:08:29.699Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTGDEV505JEYQX5MQ5Z4Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:59.697" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:29.698" grpc.time_ms=0.824 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.797301] gitaly[1382]: time="2026-06-29T22:08:29.704Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:29.703" grpc.time_ms=0.551 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 212.860604] gitaly[1382]: time="2026-06-29T22:08:29.767Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:29.767" grpc.time_ms=0.452 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 212.871875] gitaly[1382]: time="2026-06-29T22:08:29.778Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 diskcache=a4dacdcc-52c1-48b6-bca5-5a01572c568b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:08:39.105" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:29.656" pid=1382 ref=refs/keep-around/c0de400263dc825e5f844f505cbeaf8de2ec47cd remote_ip="2001:db8:1::1" revision=c0de400263dc825e5f844f505cbeaf8de2ec47cd span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.873111] gitaly[1382]: time="2026-06-29T22:08:29.779Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=93 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5451 command.oublock=8 command.real_time_ms=114 command.spawn_token_fork_ms=1 command.system_time_ms=64 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T22:08:39.105" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T22:08:29.656" grpc.time_ms=122.725 pid=1382 ref=refs/keep-around/c0de400263dc825e5f844f505cbeaf8de2ec47cd remote_ip="2001:db8:1::1" revision=c0de400263dc825e5f844f505cbeaf8de2ec47cd span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.878869] gitaly[1382]: time="2026-06-29T22:08:29.786Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:08:59.784" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:29.785" grpc.time_ms=0.513 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.001649] gitaly[1382]: time="2026-06-29T22:08:29.908Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:29.908" grpc.time_ms=0.323 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 213.033127] gitaly[1382]: time="2026-06-29T22:08:29.940Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTGDEV505JEYQX5MQ5Z4Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:39.151" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:29.932" grpc.time_ms=8.109 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.045208] gitaly[1382]: time="2026-06-29T22:08:29.952Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:29.951" grpc.time_ms=0.669 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 213.092819] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c0de400263dc825e5f844f505cbeaf8de2ec47cd HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 594 vm-test-run-nixbot-gitlab> gitlab # [ 213.094579] gitaly[1382]: time="2026-06-29T22:08:30.001Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:30.001" grpc.time_ms=0.352 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 213.113455] nixbot[3139]: {"ts": "2026-06-29T22:08:30.020Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c0de400263dc825e5f844f505cbeaf8de2ec47cd \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 213.119783] nixbot[3139]: {"ts": "2026-06-29T22:08:30.027Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-c0de400263dc-8597e0d9", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 213.133058] gitaly[1382]: time="2026-06-29T22:08:30.040Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK diskcache=5076791a-dba2-4196-aa10-e5400bef8359 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:08:59.877" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:29.877" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.148947] gitaly[1382]: time="2026-06-29T22:08:30.040Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=90 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6899 command.oublock=0 command.real_time_ms=148 command.spawn_token_fork_ms=0 command.system_time_ms=71 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:08:59.877" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:29.877" grpc.time_ms=162.821 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.170184] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 441 vm-test-run-nixbot-gitlab> gitlab # [ 213.174702] nixbot[3139]: {"ts": "2026-06-29T22:08:30.070Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1ab2c6ae3a3423ee24c4c7684b2a8ce4fd6e318f \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd' vm-test-run-nixbot-gitlab> gitlab # [ 213.222830] nix-daemon[3417]: accepted connection from pid 3537, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 213.226195] nix-daemon[3417]: accepted connection from pid 3538, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 213.251772] nixbot[3139]: INFO: ::ffff:127.0.0.1:54452 - "GET /api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=c0de400263dc825e5f844f505cbeaf8de2ec47cd', in 0.03 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': [{'level': 'warning', 'message': "unable to download 'https://cache.nixos.org/…': Could not resolve hostname (6) Could not resolve host: cache.nixos.org", 'count': 1}], 'error': None, 'created_at': '2026-06-29T22:08:29.327808Z', 'started_at': '2026-06-29T22:08:30.021977Z', 'finished_at': None}] vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 4.33 seconds) vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses vm-test-run-nixbot-gitlab> gitlab # [ 213.320406] gitaly[1382]: time="2026-06-29T22:08:30.227Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTGZ6K95CGMKK8NZ1465Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:00.225" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:30.226" grpc.time_ms=0.952 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.352795] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 68 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses, in 0.10 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T22:08:29.920Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 213.423399] gitaly[1382]: time="2026-06-29T22:08:30.330Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:09:00.328" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:30.329" grpc.time_ms=1.038 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.522777] gitaly[1382]: time="2026-06-29T22:08:30.429Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:00.426" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:30.427" grpc.time_ms=1.462 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.664505] gitaly[1382]: time="2026-06-29T22:08:30.571Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:00.568" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:30.569" grpc.time_ms=1.357 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.713178] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.585Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"be603995b8bed64176e79bff","created_at":"2026-06-29T22:08:30.581Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"075b19a886b57b942bebd9d9","job_size_bytes":3,"pid":2114,"job_status":"deduplicated","message":"PipelineProcessWorker JID-be603995b8bed64176e79bff: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} vm-test-run-nixbot-gitlab> gitlab # [ 213.714825] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.589Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"075b19a886b57b942bebd9d9","created_at":"2026-06-29T22:08:29.537Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-cfb71e52ac514b86","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.585Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2114,"job_status":"rescheduled","message":"PipelineProcessWorker JID-075b19a886b57b942bebd9d9: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 213.717642] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.595Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"075b19a886b57b942bebd9d9","created_at":"2026-06-29T22:08:29.537Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-cfb71e52ac514b86","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.585Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-075b19a886b57b942bebd9d9: done: 0.958401 sec","job_status":"done","queue_duration_s":0.052527,"scheduling_latency_s":0.05257,"gitaly_calls":4,"gitaly_duration_s":0.265468,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.15654,"redis_read_bytes":267,"redis_write_bytes":24007,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002905,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.005211,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.027095,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.01323,"redis_queues_read_bytes":27,"redis_queues_write_bytes":12386,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.038985,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001629,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002309,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.062271,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.18,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.024,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.029,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.126,"db_ci_duration_s":0.055,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.229683,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.856449632,"duration_s":0.958401,"completed_at":"2026-06-29T22:08:30.595Z","load_balancing_strategy":"primary","db_duration_s":0.186677,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 213.823065] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.598Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"03ba8ea2aa9beeb83f07c11b","created_at":"2026-06-29T22:08:29.792Z","trace_propagation_headers":{"sentry-trace":"f9233e3c53ec40c293e259b272aab5f5-b6d28b56169a4f53","baggage":"sentry-trace_id=f9233e3c53ec40c293e259b272aab5f5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTG6XZQP65FTR299778Z9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.796Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-03ba8ea2aa9beeb83f07c11b: start","job_status":"start","queue_duration_s":0.801104,"scheduling_latency_s":0.801128} vm-test-run-nixbot-gitlab> gitlab # [ 213.824355] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.622Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"03ba8ea2aa9beeb83f07c11b","created_at":"2026-06-29T22:08:29.792Z","trace_propagation_headers":{"sentry-trace":"f9233e3c53ec40c293e259b272aab5f5-b6d28b56169a4f53","baggage":"sentry-trace_id=f9233e3c53ec40c293e259b272aab5f5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTG6XZQP65FTR299778Z9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.796Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-03ba8ea2aa9beeb83f07c11b: done: 0.024893 sec","job_status":"done","queue_duration_s":0.801104,"scheduling_latency_s":0.801128,"redis_calls":7,"redis_duration_s":0.002834,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002046,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024893,"completed_at":"2026-06-29T22:08:30.622Z","load_balancing_strategy":"primary","db_duration_s":0.004815,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 213.902136] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.624Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a43842de233995102133b881","created_at":"2026-06-29T22:08:29.797Z","trace_propagation_headers":{"sentry-trace":"f9233e3c53ec40c293e259b272aab5f5-b6d28b56169a4f53","baggage":"sentry-trace_id=f9233e3c53ec40c293e259b272aab5f5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTG6XZQP65FTR299778Z9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.800Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a43842de233995102133b881: start","job_status":"start","queue_duration_s":0.824922,"scheduling_latency_s":0.824944} vm-test-run-nixbot-gitlab> gitlab # [ 213.905196] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.678Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a43842de233995102133b881","created_at":"2026-06-29T22:08:29.797Z","trace_propagation_headers":{"sentry-trace":"f9233e3c53ec40c293e259b272aab5f5-b6d28b56169a4f53","baggage":"sentry-trace_id=f9233e3c53ec40c293e259b272aab5f5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTG6XZQP65FTR299778Z9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.800Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a43842de233995102133b881: done: 0.053459 sec","job_status":"done","queue_duration_s":0.824922,"scheduling_latency_s":0.824944,"redis_calls":10,"redis_duration_s":0.004123,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001908,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053459,"completed_at":"2026-06-29T22:08:30.678Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017234,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 213.980290] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.680Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"10719163e013f8cc38986be8","created_at":"2026-06-29T22:08:29.840Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.853Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-10719163e013f8cc38986be8: start","job_status":"start","queue_duration_s":0.826442,"scheduling_latency_s":0.826466} vm-test-run-nixbot-gitlab> gitlab # [ 214.059718] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.774Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"10719163e013f8cc38986be8","created_at":"2026-06-29T22:08:29.840Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.853Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-10719163e013f8cc38986be8: done: 0.093719 sec","job_status":"done","queue_duration_s":0.826442,"scheduling_latency_s":0.826466,"redis_calls":15,"redis_duration_s":0.005825,"redis_read_bytes":13,"redis_write_bytes":3009,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00075,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000492,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1624,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003003,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000821,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000759,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066151,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000613486,"duration_s":0.093719,"completed_at":"2026-06-29T22:08:30.774Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01815,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 214.139067] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.776Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"b2d8d47553f53439de1ddca3","created_at":"2026-06-29T22:08:29.933Z","trace_propagation_headers":{"sentry-trace":"f9233e3c53ec40c293e259b272aab5f5-b6d28b56169a4f53","baggage":"sentry-trace_id=f9233e3c53ec40c293e259b272aab5f5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTG6XZQP65FTR299778Z9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.941Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b2d8d47553f53439de1ddca3: start","job_status":"start","queue_duration_s":0.834251,"scheduling_latency_s":0.834274} vm-test-run-nixbot-gitlab> gitlab # [ 214.140312] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.920Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"b2d8d47553f53439de1ddca3","created_at":"2026-06-29T22:08:29.933Z","trace_propagation_headers":{"sentry-trace":"f9233e3c53ec40c293e259b272aab5f5-b6d28b56169a4f53","baggage":"sentry-trace_id=f9233e3c53ec40c293e259b272aab5f5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTG6XZQP65FTR299778Z9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:29.941Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b2d8d47553f53439de1ddca3: done: 0.14488 sec","job_status":"done","queue_duration_s":0.834251,"scheduling_latency_s":0.834274,"gitaly_calls":3,"gitaly_duration_s":0.011037,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.013307,"redis_read_bytes":99,"redis_write_bytes":6012,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001576,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003511,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000462,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1131,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002948,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000417,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000821,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001996,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086728,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.119658504,"duration_s":0.14488,"completed_at":"2026-06-29T22:08:30.920Z","load_balancing_strategy":"primary","db_duration_s":0.042193,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 214.229292] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.923Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6ab09cec4ad8d0f709c0caef","created_at":"2026-06-29T22:08:30.070Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.096Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6ab09cec4ad8d0f709c0caef: start","job_status":"start","queue_duration_s":0.826441,"scheduling_latency_s":0.826466} vm-test-run-nixbot-gitlab> gitlab # [ 214.230827] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.952Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6ab09cec4ad8d0f709c0caef","created_at":"2026-06-29T22:08:30.070Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.096Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6ab09cec4ad8d0f709c0caef: done: 0.029634 sec","job_status":"done","queue_duration_s":0.826441,"scheduling_latency_s":0.826466,"redis_calls":10,"redis_duration_s":0.003223,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000561,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000633,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000377422,"duration_s":0.029634,"completed_at":"2026-06-29T22:08:30.952Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses vm-test-run-nixbot-gitlab> gitlab # [ 214.309053] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.954Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"5f09353c173ace3d9d94db2c","created_at":"2026-06-29T22:08:30.764Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-76f972ffed514fde","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:0ca5bfafd3ade16523e350112b03cd2e2dc5d48980a0ef972b82f82ab3f88102","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.766Z","job_size_bytes":464,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-5f09353c173ace3d9d94db2c: start","job_status":"start","queue_duration_s":0.187723,"scheduling_latency_s":0.187746} vm-test-run-nixbot-gitlab> gitlab # [ 214.310831] gitaly[1382]: time="2026-06-29T22:08:30.836Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:09:00.835" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:30.835" grpc.time_ms=1.192 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.384521] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T22:08:30.993Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"5f09353c173ace3d9d94db2c","created_at":"2026-06-29T22:08:30.764Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-76f972ffed514fde","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:0ca5bfafd3ade16523e350112b03cd2e2dc5d48980a0ef972b82f82ab3f88102","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.766Z","job_size_bytes":464,"pid":2114,"sidekiq_tid":"m476","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-5f09353c173ace3d9d94db2c: fail: 0.038571 sec","job_status":"fail","queue_duration_s":0.187723,"scheduling_latency_s":0.187746,"redis_calls":7,"redis_duration_s":0.001718,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001255,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038571,"completed_at":"2026-06-29T22:08:30.992Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.008842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 214.938424] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:30.996Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9fe87eb93280f70b353b2dcd","created_at":"2026-06-29T22:08:30.213Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.240Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9fe87eb93280f70b353b2dcd: start","job_status":"start","queue_duration_s":0.756619,"scheduling_latency_s":0.756674} vm-test-run-nixbot-gitlab> gitlab # [ 215.011449] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.095Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9fe87eb93280f70b353b2dcd","created_at":"2026-06-29T22:08:30.213Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.240Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9fe87eb93280f70b353b2dcd: done: 0.098975 sec","job_status":"done","queue_duration_s":0.756619,"scheduling_latency_s":0.756674,"redis_calls":10,"redis_duration_s":0.0027229999999999997,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000616,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001566,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000541,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000421841,"duration_s":0.098975,"completed_at":"2026-06-29T22:08:31.095Z","load_balancing_strategy":"primary","db_duration_s":0.064805,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.088753] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.097Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"6f4dda7e5948f48e537f5110","created_at":"2026-06-29T22:08:30.268Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.277Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-6f4dda7e5948f48e537f5110: start","job_status":"start","queue_duration_s":0.819624,"scheduling_latency_s":0.819646} vm-test-run-nixbot-gitlab> gitlab # [ 215.090360] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.124Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"6f4dda7e5948f48e537f5110","created_at":"2026-06-29T22:08:30.268Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.277Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-6f4dda7e5948f48e537f5110: done: 0.027705 sec","job_status":"done","queue_duration_s":0.819624,"scheduling_latency_s":0.819646,"redis_calls":13,"redis_duration_s":0.002898,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000433,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001397,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000627,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000389435,"duration_s":0.027705,"completed_at":"2026-06-29T22:08:31.124Z","load_balancing_strategy":"primary","db_duration_s":0.003457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.191388] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.126Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c356318801000cfacca9c31f","created_at":"2026-06-29T22:08:30.278Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.285Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c356318801000cfacca9c31f: start","job_status":"start","queue_duration_s":0.840248,"scheduling_latency_s":0.840269} vm-test-run-nixbot-gitlab> gitlab # [ 215.193478] gitaly[1382]: time="2026-06-29T22:08:30.876Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:00.874" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:30.875" grpc.time_ms=0.781 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.195462] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.150Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c356318801000cfacca9c31f","created_at":"2026-06-29T22:08:30.278Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.285Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c356318801000cfacca9c31f: done: 0.024241 sec","job_status":"done","queue_duration_s":0.840248,"scheduling_latency_s":0.840269,"redis_calls":10,"redis_duration_s":0.002471,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000677,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000333283,"duration_s":0.024241,"completed_at":"2026-06-29T22:08:31.150Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.300528] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.152Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"72a9967f66c7cb7be2a5f247","created_at":"2026-06-29T22:08:30.287Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.297Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-72a9967f66c7cb7be2a5f247: start","job_status":"start","queue_duration_s":0.855235,"scheduling_latency_s":0.855288} vm-test-run-nixbot-gitlab> gitlab # [ 215.302384] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.226Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"72a9967f66c7cb7be2a5f247","created_at":"2026-06-29T22:08:30.287Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.297Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-72a9967f66c7cb7be2a5f247: done: 0.073593 sec","job_status":"done","queue_duration_s":0.855235,"scheduling_latency_s":0.855288,"redis_calls":11,"redis_duration_s":0.003058,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000476,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002099,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000483,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.041,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000426311,"duration_s":0.073593,"completed_at":"2026-06-29T22:08:31.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.403594] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.227Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b4c7f333cc80acdf0006429b","created_at":"2026-06-29T22:08:30.300Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.308Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b4c7f333cc80acdf0006429b: start","job_status":"start","queue_duration_s":0.919251,"scheduling_latency_s":0.919276} vm-test-run-nixbot-gitlab> gitlab # [ 215.405228] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.252Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b4c7f333cc80acdf0006429b","created_at":"2026-06-29T22:08:30.300Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.308Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b4c7f333cc80acdf0006429b: done: 0.024803 sec","job_status":"done","queue_duration_s":0.919251,"scheduling_latency_s":0.919276,"redis_calls":10,"redis_duration_s":0.002559,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001611,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000371276,"duration_s":0.024803,"completed_at":"2026-06-29T22:08:31.252Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002887,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.499981] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.253Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"6e5f8060e19ec38c1103650d","created_at":"2026-06-29T22:08:30.313Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.316Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6e5f8060e19ec38c1103650d: start","job_status":"start","queue_duration_s":0.937209,"scheduling_latency_s":0.93723} vm-test-run-nixbot-gitlab> gitlab # [ 215.501378] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.280Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"6e5f8060e19ec38c1103650d","created_at":"2026-06-29T22:08:30.313Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.316Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6e5f8060e19ec38c1103650d: done: 0.026107 sec","job_status":"done","queue_duration_s":0.937209,"scheduling_latency_s":0.93723,"redis_calls":7,"redis_duration_s":0.001741,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001219,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026107,"completed_at":"2026-06-29T22:08:31.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.588340] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.281Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"c89c3b3610c31f4139750943","created_at":"2026-06-29T22:08:30.475Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.497Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c89c3b3610c31f4139750943: start","job_status":"start","queue_duration_s":0.783692,"scheduling_latency_s":0.783714} vm-test-run-nixbot-gitlab> gitlab # [ 215.589860] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.313Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"c89c3b3610c31f4139750943","created_at":"2026-06-29T22:08:30.475Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.497Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c89c3b3610c31f4139750943: done: 0.031971 sec","job_status":"done","queue_duration_s":0.783692,"scheduling_latency_s":0.783714,"redis_calls":10,"redis_duration_s":0.002583,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000519,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001548,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000516,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000419886,"duration_s":0.031971,"completed_at":"2026-06-29T22:08:31.313Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.664716] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.315Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"45dbce262858003eae600287","created_at":"2026-06-29T22:08:30.500Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.521Z","job_size_bytes":6,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-45dbce262858003eae600287: start","job_status":"start","queue_duration_s":0.79428,"scheduling_latency_s":0.794302} vm-test-run-nixbot-gitlab> gitlab # [ 215.666388] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.336Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"45dbce262858003eae600287","created_at":"2026-06-29T22:08:30.500Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.521Z","job_size_bytes":6,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-45dbce262858003eae600287: done: 0.021685 sec","job_status":"done","queue_duration_s":0.79428,"scheduling_latency_s":0.794302,"redis_calls":10,"redis_duration_s":0.002405,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000492,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001419,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000494,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349765,"duration_s":0.021685,"completed_at":"2026-06-29T22:08:31.336Z","load_balancing_strategy":"primary","db_duration_s":0.00168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.770174] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.338Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"49b774737e05b85911b1862f","created_at":"2026-06-29T22:08:30.589Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.590Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-49b774737e05b85911b1862f: start","job_status":"start","queue_duration_s":0.74755,"scheduling_latency_s":0.74757} vm-test-run-nixbot-gitlab> gitlab # [ 215.772109] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.414Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"49b774737e05b85911b1862f","created_at":"2026-06-29T22:08:30.589Z","trace_propagation_headers":{"sentry-trace":"d072a0d0ed904edfb2d6804a88aa3433-d58bd983961e472b","baggage":"sentry-trace_id=d072a0d0ed904edfb2d6804a88aa3433,sentry-release=Unknown"},"correlation_id":"01KWAPTG7DHZQ8VXAQHB199FVK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.590Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-49b774737e05b85911b1862f: done: 0.076241 sec","job_status":"done","queue_duration_s":0.74755,"scheduling_latency_s":0.74757,"gitaly_calls":2,"gitaly_duration_s":0.007048,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003623,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000888,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001369,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000276,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.054480667,"duration_s":0.076241,"completed_at":"2026-06-29T22:08:31.414Z","load_balancing_strategy":"primary","db_duration_s":0.01363,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 215.898177] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.416Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b42721453d78087d9326bf25","created_at":"2026-06-29T22:08:30.823Z","trace_propagation_headers":{"sentry-trace":"f9233e3c53ec40c293e259b272aab5f5-7115e5f69aff44c2","baggage":"sentry-trace_id=f9233e3c53ec40c293e259b272aab5f5,sentry-release=Unknown"},"correlation_id":"01KWAPTG6XZQP65FTR299778Z9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.825Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b42721453d78087d9326bf25: start","job_status":"start","queue_duration_s":0.590581,"scheduling_latency_s":0.590603} vm-test-run-nixbot-gitlab> gitlab # [ 215.899721] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:31.441Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b42721453d78087d9326bf25","created_at":"2026-06-29T22:08:30.823Z","trace_propagation_headers":{"sentry-trace":"f9233e3c53ec40c293e259b272aab5f5-7115e5f69aff44c2","baggage":"sentry-trace_id=f9233e3c53ec40c293e259b272aab5f5,sentry-release=Unknown"},"correlation_id":"01KWAPTG6XZQP65FTR299778Z9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:30.825Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b42721453d78087d9326bf25: done: 0.025072 sec","job_status":"done","queue_duration_s":0.590581,"scheduling_latency_s":0.590603,"redis_calls":7,"redis_duration_s":0.001601,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025072,"completed_at":"2026-06-29T22:08:31.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003029,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.926434] gitaly[1382]: time="2026-06-29T22:08:30.908Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG6XZQP65FTR299778Z9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:00.007" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:30.907" grpc.time_ms=0.57 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.945927] gitaly[1382]: time="2026-06-29T22:08:31.376Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:09:01.374" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:31.374" grpc.time_ms=1.114 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.027323] gitaly[1382]: time="2026-06-29T22:08:31.403Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTG7DHZQ8VXAQHB199FVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:01.401" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:31.402" grpc.time_ms=0.762 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.181486] gitaly[1382]: time="2026-06-29T22:08:33.088Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTKQ9T750Z8CJS7F7WEEB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:03.085" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:33.086" grpc.time_ms=1.484 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.268972] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 173 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses, in 1.94 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T22:08:29.920Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses vm-test-run-nixbot-gitlab> gitlab # [ 217.389497] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:34.295Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"4cada36912af316a0209476e","created_at":"2026-06-29T22:08:15.968Z","trace_propagation_headers":{"sentry-trace":"b4ac1def9b584470b63b03b834cd904b-1d2c1297a89d4fb2","baggage":"sentry-trace_id=b4ac1def9b584470b63b03b834cd904b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT2XQK6S2FDP8H5R5C3VM","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:1aaa7278415282300418518b09ab8b598ab9223c072a7d0cc495005e326e8d8b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:34.288Z","retry_count":0,"job_size_bytes":451,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-4cada36912af316a0209476e: start","job_status":"start","queue_duration_s":0.006478,"scheduling_latency_s":0.006518} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses, in 0.26 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T22:08:29.920Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 217.419057] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T22:08:34.325Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"4cada36912af316a0209476e","created_at":"2026-06-29T22:08:15.968Z","trace_propagation_headers":{"sentry-trace":"b4ac1def9b584470b63b03b834cd904b-1d2c1297a89d4fb2","baggage":"sentry-trace_id=b4ac1def9b584470b63b03b834cd904b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPT2XQK6S2FDP8H5R5C3VM","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:1aaa7278415282300418518b09ab8b598ab9223c072a7d0cc495005e326e8d8b","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:34.288Z","retry_count":0,"job_size_bytes":451,"pid":2114,"sidekiq_tid":"ma3q","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-4cada36912af316a0209476e: fail: 0.030325 sec","job_status":"fail","queue_duration_s":0.006478,"scheduling_latency_s":0.006518,"redis_calls":7,"redis_duration_s":0.002234,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001564,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030325,"completed_at":"2026-06-29T22:08:34.325Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.004088,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.948002] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 101 vm-test-run-nixbot-gitlab> gitlab # [ 217.951337] nix-daemon[3417]: accepted connection from pid 3570, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 217.953093] gitaly[1382]: time="2026-06-29T22:08:34.326Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTMYCDV2CNEYBKXS7W8NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:04.024" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:34.325" grpc.time_ms=0.666 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.091705] gitaly[1382]: time="2026-06-29T22:08:34.998Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:04.696" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:34.997" grpc.time_ms=1.091 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.151992] gitaly[1382]: time="2026-06-29T22:08:35.059Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:45.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:35.047" grpc.time_ms=11.252 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.186311] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.092Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d36ceffb8f418c72f1f27861","created_at":"2026-06-29T22:08:35.086Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-2054e514e85947fb","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.089Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d36ceffb8f418c72f1f27861: start","job_status":"start","queue_duration_s":0.002943,"scheduling_latency_s":0.003024} vm-test-run-nixbot-gitlab> gitlab # [ 218.212813] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c0de400263dc825e5f844f505cbeaf8de2ec47cd HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 149 vm-test-run-nixbot-gitlab> gitlab # [ 218.217709] nixbot[3139]: {"ts": "2026-06-29T22:08:35.103Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c0de400263dc825e5f844f505cbeaf8de2ec47cd \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 218.269914] gitaly[1382]: time="2026-06-29T22:08:35.177Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTNS2J145PQ35YQP1P11K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:05.175" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:35.176" grpc.time_ms=0.864 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.306122] gitaly[1382]: time="2026-06-29T22:08:35.213Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTNS2J145PQ35YQP1P11K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:45.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:35.206" grpc.time_ms=6.233 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.324446] gitaly[1382]: time="2026-06-29T22:08:35.231Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:35.231" grpc.time_ms=0.377 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 218.361729] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c0de400263dc825e5f844f505cbeaf8de2ec47cd HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 162 vm-test-run-nixbot-gitlab> gitlab # [ 218.366888] nixbot[3139]: {"ts": "2026-06-29T22:08:35.274Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c0de400263dc825e5f844f505cbeaf8de2ec47cd \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 218.370192] gitaly[1382]: time="2026-06-29T22:08:35.277Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:35.276" grpc.time_ms=0.482 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 218.415961] gitaly[1382]: time="2026-06-29T22:08:35.323Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:35.322" grpc.time_ms=0.443 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 218.441696] gitaly[1382]: time="2026-06-29T22:08:35.348Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 diskcache=a47bd3a3-b354-4418-83cd-eecba7c8e353 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:09:05.193" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:35.194" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.442408] gitaly[1382]: time="2026-06-29T22:08:35.349Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=120 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6838 command.oublock=0 command.real_time_ms=145 command.spawn_token_fork_ms=1 command.system_time_ms=91 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:09:05.193" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:35.194" grpc.time_ms=154.497 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses vm-test-run-nixbot-gitlab> gitlab # [ 218.539158] gitaly[1382]: time="2026-06-29T22:08:35.446Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:09:05.443" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:35.444" grpc.time_ms=1.624 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.574737] gitaly[1382]: time="2026-06-29T22:08:35.481Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:05.480" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:35.480" grpc.time_ms=1.106 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.638349] gitaly[1382]: time="2026-06-29T22:08:35.545Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:05.543" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:35.544" grpc.time_ms=1.058 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.669606] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.575Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"91bfac6f4d33f8943a407357","created_at":"2026-06-29T22:08:35.573Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"d36ceffb8f418c72f1f27861","job_size_bytes":3,"pid":2114,"job_status":"deduplicated","message":"PipelineProcessWorker JID-91bfac6f4d33f8943a407357: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} vm-test-run-nixbot-gitlab> gitlab # [ 218.695292] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.602Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d36ceffb8f418c72f1f27861","created_at":"2026-06-29T22:08:35.086Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-2054e514e85947fb","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.089Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2114,"job_status":"rescheduled","message":"PipelineProcessWorker JID-d36ceffb8f418c72f1f27861: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 218.731663] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.637Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d36ceffb8f418c72f1f27861","created_at":"2026-06-29T22:08:35.086Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-2054e514e85947fb","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.089Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d36ceffb8f418c72f1f27861: done: 0.545377 sec","job_status":"done","queue_duration_s":0.002943,"scheduling_latency_s":0.003024,"gitaly_calls":4,"gitaly_duration_s":0.250103,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.081142,"redis_read_bytes":262,"redis_write_bytes":23382,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001677,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003976,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00147,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.00718,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14463,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.059159,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000282,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001737,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003984,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.155799,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.444376031,"duration_s":0.545377,"completed_at":"2026-06-29T22:08:35.637Z","load_balancing_strategy":"primary","db_duration_s":0.055809,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 218.820811] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.727Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"72ff8559067dc1444f66cfdf","created_at":"2026-06-29T22:08:35.184Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.186Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-72ff8559067dc1444f66cfdf: start","job_status":"start","queue_duration_s":0.540809,"scheduling_latency_s":0.54088} vm-test-run-nixbot-gitlab> gitlab # [ 218.850500] gitaly[1382]: time="2026-06-29T22:08:35.757Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTP627AQB3XCS02X7MD8M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:05.454" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:35.755" grpc.time_ms=1.361 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.897851] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 282 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses, in 0.34 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T22:08:29.920Z', 'started_at': None, 'finished_at': '2026-06-29T22:08:35.077Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T22:08:35.231Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 218.905389] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.811Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"72ff8559067dc1444f66cfdf","created_at":"2026-06-29T22:08:35.184Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.186Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-72ff8559067dc1444f66cfdf: done: 0.084836 sec","job_status":"done","queue_duration_s":0.540809,"scheduling_latency_s":0.54088,"redis_calls":7,"redis_duration_s":0.005263,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003967,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001296,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.084836,"completed_at":"2026-06-29T22:08:35.811Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013242,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 218.908400] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.815Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"47e0581d16fb1fe452970157","created_at":"2026-06-29T22:08:35.390Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.393Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-47e0581d16fb1fe452970157: start","job_status":"start","queue_duration_s":0.421965,"scheduling_latency_s":0.422015} vm-test-run-nixbot-gitlab> gitlab # [ 218.986673] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.843Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"47e0581d16fb1fe452970157","created_at":"2026-06-29T22:08:35.390Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.393Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-47e0581d16fb1fe452970157: done: 0.028364 sec","job_status":"done","queue_duration_s":0.421965,"scheduling_latency_s":0.422015,"redis_calls":7,"redis_duration_s":0.003866,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028364,"completed_at":"2026-06-29T22:08:35.843Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005199,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.058199] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.845Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"165269f5860c1d44f9c87163","created_at":"2026-06-29T22:08:35.402Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.407Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-165269f5860c1d44f9c87163: start","job_status":"start","queue_duration_s":0.437854,"scheduling_latency_s":0.437877} vm-test-run-nixbot-gitlab> gitlab # [ 219.074898] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.981Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"165269f5860c1d44f9c87163","created_at":"2026-06-29T22:08:35.402Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.407Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-165269f5860c1d44f9c87163: done: 0.135944 sec","job_status":"done","queue_duration_s":0.437854,"scheduling_latency_s":0.437877,"redis_calls":7,"redis_duration_s":0.0037679999999999996,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002728,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.105,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.105,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.135944,"completed_at":"2026-06-29T22:08:35.981Z","load_balancing_strategy":"primary","db_duration_s":0.104809,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.089273] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:35.985Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"554d515001a5f8c72c2f7e02","created_at":"2026-06-29T22:08:35.420Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.423Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-554d515001a5f8c72c2f7e02: start","job_status":"start","queue_duration_s":0.561948,"scheduling_latency_s":0.562004} vm-test-run-nixbot-gitlab> gitlab # [ 219.158345] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.033Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"554d515001a5f8c72c2f7e02","created_at":"2026-06-29T22:08:35.420Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.423Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-554d515001a5f8c72c2f7e02: done: 0.048953 sec","job_status":"done","queue_duration_s":0.561948,"scheduling_latency_s":0.562004,"redis_calls":10,"redis_duration_s":0.006066,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003283,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001703,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048953,"completed_at":"2026-06-29T22:08:36.033Z","load_balancing_strategy":"primary","db_duration_s":0.006646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.229429] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.035Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"8ffd45e5677a9973cbf0e09b","created_at":"2026-06-29T22:08:35.424Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.426Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-8ffd45e5677a9973cbf0e09b: start","job_status":"start","queue_duration_s":0.60918,"scheduling_latency_s":0.609204} vm-test-run-nixbot-gitlab> gitlab # [ 219.230992] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.059Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"8ffd45e5677a9973cbf0e09b","created_at":"2026-06-29T22:08:35.424Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.426Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-8ffd45e5677a9973cbf0e09b: done: 0.023483 sec","job_status":"done","queue_duration_s":0.60918,"scheduling_latency_s":0.609204,"redis_calls":7,"redis_duration_s":0.003241,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023483,"completed_at":"2026-06-29T22:08:36.059Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006174,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.276990] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.061Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"46be8661e700a92135328cf1","created_at":"2026-06-29T22:08:35.427Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.431Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-46be8661e700a92135328cf1: start","job_status":"start","queue_duration_s":0.629501,"scheduling_latency_s":0.629522} vm-test-run-nixbot-gitlab> gitlab # [ 219.355681] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.084Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"46be8661e700a92135328cf1","created_at":"2026-06-29T22:08:35.427Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.431Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-46be8661e700a92135328cf1: done: 0.023341 sec","job_status":"done","queue_duration_s":0.629501,"scheduling_latency_s":0.629522,"redis_calls":8,"redis_duration_s":0.003301,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002535,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000766,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023341,"completed_at":"2026-06-29T22:08:36.084Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.370816] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.086Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9d24e390aba519717ad76106","created_at":"2026-06-29T22:08:35.433Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.434Z","job_size_bytes":66,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9d24e390aba519717ad76106: start","job_status":"start","queue_duration_s":0.651852,"scheduling_latency_s":0.651876} vm-test-run-nixbot-gitlab> gitlab # [ 219.456861] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.167Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9d24e390aba519717ad76106","created_at":"2026-06-29T22:08:35.433Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.434Z","job_size_bytes":66,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9d24e390aba519717ad76106: done: 0.080718 sec","job_status":"done","queue_duration_s":0.651852,"scheduling_latency_s":0.651876,"redis_calls":7,"redis_duration_s":0.003131,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002336,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000795,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.080718,"completed_at":"2026-06-29T22:08:36.167Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003202,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.528381] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.169Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"56384e2ed61e44194b3cb5c3","created_at":"2026-06-29T22:08:35.435Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.437Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-56384e2ed61e44194b3cb5c3: start","job_status":"start","queue_duration_s":0.732118,"scheduling_latency_s":0.732145} vm-test-run-nixbot-gitlab> gitlab # [ 219.530115] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.197Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"56384e2ed61e44194b3cb5c3","created_at":"2026-06-29T22:08:35.435Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.437Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-56384e2ed61e44194b3cb5c3: done: 0.028577 sec","job_status":"done","queue_duration_s":0.732118,"scheduling_latency_s":0.732145,"redis_calls":7,"redis_duration_s":0.002972,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000868,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028577,"completed_at":"2026-06-29T22:08:36.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.615683] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.199Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"964fc57ffe0b7e3766b50250","created_at":"2026-06-29T22:08:35.509Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.511Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-964fc57ffe0b7e3766b50250: start","job_status":"start","queue_duration_s":0.688807,"scheduling_latency_s":0.68883} vm-test-run-nixbot-gitlab> gitlab # [ 219.617365] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.223Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"964fc57ffe0b7e3766b50250","created_at":"2026-06-29T22:08:35.509Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.511Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-964fc57ffe0b7e3766b50250: done: 0.023144 sec","job_status":"done","queue_duration_s":0.688807,"scheduling_latency_s":0.68883,"redis_calls":7,"redis_duration_s":0.0031910000000000003,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002363,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023144,"completed_at":"2026-06-29T22:08:36.223Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.703561] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.225Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e87b738769cc03ad5a94b2c2","created_at":"2026-06-29T22:08:35.511Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.513Z","job_size_bytes":6,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e87b738769cc03ad5a94b2c2: start","job_status":"start","queue_duration_s":0.711982,"scheduling_latency_s":0.712003} vm-test-run-nixbot-gitlab> gitlab # [ 219.705246] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.240Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e87b738769cc03ad5a94b2c2","created_at":"2026-06-29T22:08:35.511Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.513Z","job_size_bytes":6,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e87b738769cc03ad5a94b2c2: done: 0.015688 sec","job_status":"done","queue_duration_s":0.711982,"scheduling_latency_s":0.712003,"redis_calls":7,"redis_duration_s":0.0034649999999999998,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002165,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0013,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015688,"completed_at":"2026-06-29T22:08:36.240Z","load_balancing_strategy":"primary","db_duration_s":0.000776,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.786453] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.242Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"840775eaca0660c8cde8f411","created_at":"2026-06-29T22:08:35.513Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.517Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-840775eaca0660c8cde8f411: start","job_status":"start","queue_duration_s":0.724872,"scheduling_latency_s":0.724927} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses vm-test-run-nixbot-gitlab> gitlab # [ 219.788755] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.274Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"840775eaca0660c8cde8f411","created_at":"2026-06-29T22:08:35.513Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.517Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-840775eaca0660c8cde8f411: done: 0.032556 sec","job_status":"done","queue_duration_s":0.724872,"scheduling_latency_s":0.724927,"redis_calls":10,"redis_duration_s":0.004564,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000767,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002937,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00086,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000515429,"duration_s":0.032556,"completed_at":"2026-06-29T22:08:36.274Z","load_balancing_strategy":"primary","db_duration_s":0.00835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 219.882105] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.277Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"9fd9d6ee6eb288ff16946786","created_at":"2026-06-29T22:08:35.602Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.628Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9fd9d6ee6eb288ff16946786: start","job_status":"start","queue_duration_s":0.648675,"scheduling_latency_s":0.648696} vm-test-run-nixbot-gitlab> gitlab # [ 219.883412] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.425Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"9fd9d6ee6eb288ff16946786","created_at":"2026-06-29T22:08:35.602Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-30932314d4064880","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:35.628Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9fd9d6ee6eb288ff16946786: done: 0.148072 sec","job_status":"done","queue_duration_s":0.648675,"scheduling_latency_s":0.648696,"gitaly_calls":3,"gitaly_duration_s":0.010601,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.014524,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001618,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004275,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000486,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003148,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00047,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000982,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001927,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084258,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.122042886,"duration_s":0.148072,"completed_at":"2026-06-29T22:08:36.425Z","load_balancing_strategy":"primary","db_duration_s":0.048075,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 219.963446] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.427Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"3bc47248631092492a8f9120","created_at":"2026-06-29T22:08:36.331Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-7f39c02564e548d0","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:36.333Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3bc47248631092492a8f9120: start","job_status":"start","queue_duration_s":0.093541,"scheduling_latency_s":0.093565} vm-test-run-nixbot-gitlab> gitlab # [ 220.039623] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:36.456Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"3bc47248631092492a8f9120","created_at":"2026-06-29T22:08:36.331Z","trace_propagation_headers":{"sentry-trace":"6a2b22b4ea7544b3b56d35c6f1032b5c-7f39c02564e548d0","baggage":"sentry-trace_id=6a2b22b4ea7544b3b56d35c6f1032b5c,sentry-release=Unknown"},"correlation_id":"01KWAPTNM74W3J3G6V5JA6W364","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:36.333Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3bc47248631092492a8f9120: done: 0.029318 sec","job_status":"done","queue_duration_s":0.093541,"scheduling_latency_s":0.093565,"redis_calls":7,"redis_duration_s":0.003393,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002222,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001171,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029318,"completed_at":"2026-06-29T22:08:36.456Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 220.046161] gitaly[1382]: time="2026-06-29T22:08:36.344Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:09:06.342" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:36.343" grpc.time_ms=0.965 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.121214] gitaly[1382]: time="2026-06-29T22:08:36.378Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:06.377" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:36.377" grpc.time_ms=0.985 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.122601] gitaly[1382]: time="2026-06-29T22:08:36.411Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTNM74W3J3G6V5JA6W364 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:06.409" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:36.409" grpc.time_ms=0.978 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.176236] gitaly[1382]: time="2026-06-29T22:08:37.083Z" level=info msg="finished unary call with code OK" catfile.duration_ms=36 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=36 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTQFV47FJQTSDZ6J9Z7DF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:07.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:37.046" grpc.time_ms=36.792 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses, in 0.30 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T22:08:29.920Z', 'started_at': None, 'finished_at': '2026-06-29T22:08:35.077Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T22:08:35.231Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 220.202293] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 249 vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses vm-test-run-nixbot-gitlab> gitlab # [ 221.326854] gitaly[1382]: time="2026-06-29T22:08:38.233Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTRRGF5DQFRX78687MYV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:08.230" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:38.231" grpc.time_ms=1.397 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 221.373188] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 118 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses, in 0.17 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T22:08:29.920Z', 'started_at': None, 'finished_at': '2026-06-29T22:08:35.077Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T22:08:35.231Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 222.256389] nix-daemon[3417]: accepted connection from pid 3641, user nixbot vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses vm-test-run-nixbot-gitlab> gitlab # [ 222.383559] gitaly[1382]: time="2026-06-29T22:08:39.289Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:09.286" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:39.287" grpc.time_ms=1.349 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 222.462392] gitaly[1382]: time="2026-06-29T22:08:39.369Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T22:08:49.159" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:39.360" grpc.time_ms=8.892 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 222.479132] gitaly[1382]: time="2026-06-29T22:08:39.386Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSX08K9DSQAV10QWK2TQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:09.084" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:39.385" grpc.time_ms=0.943 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 222.520136] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 97 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses, in 0.15 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T22:08:29.920Z', 'started_at': None, 'finished_at': '2026-06-29T22:08:35.077Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T22:08:35.231Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 222.554706] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c0de400263dc825e5f844f505cbeaf8de2ec47cd HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 260 vm-test-run-nixbot-gitlab> gitlab # [ 222.559894] nixbot[3139]: {"ts": "2026-06-29T22:08:39.463Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c0de400263dc825e5f844f505cbeaf8de2ec47cd \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 222.634622] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:39.541Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"19d1038ff113ee6347af6d9e","created_at":"2026-06-29T22:08:39.443Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-e830e5306e1c40fb","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.446Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-19d1038ff113ee6347af6d9e: start","job_status":"start","queue_duration_s":0.094322,"scheduling_latency_s":0.094363} vm-test-run-nixbot-gitlab> gitlab # [ 222.781849] gitaly[1382]: time="2026-06-29T22:08:39.689Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:39.688" grpc.time_ms=0.533 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 222.784315] nix-daemon[3417]: accepted connection from pid 3662, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 222.846109] gitaly[1382]: time="2026-06-29T22:08:39.753Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:39.752" grpc.time_ms=0.436 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 222.885131] gitaly[1382]: time="2026-06-29T22:08:39.792Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T22:08:39.791" grpc.time_ms=0.472 pid=1382 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 222.903847] gitaly[1382]: time="2026-06-29T22:08:39.811Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:09:09.655" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:39.655" operations="[repack_refs]" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 222.920654] gitaly[1382]: time="2026-06-29T22:08:39.827Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM diskcache=bf8e46f9-0cf0-4f72-af71-5aaac4fb5995 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:09:09.655" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:39.655" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 222.922413] gitaly[1382]: time="2026-06-29T22:08:39.828Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=125 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7092 command.oublock=0 command.real_time_ms=153 command.spawn_token_fork_ms=0 command.system_time_ms=96 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:09:09.655" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:39.655" grpc.time_ms=172.079 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 222.924099] gitaly[1382]: time="2026-06-29T22:08:39.831Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:09:09.655" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:39.655" pid=1382 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-06-29 22:08:17.441983107 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 222.927835] gitaly[1382]: time="2026-06-29T22:08:39.834Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=7 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTTB2MZ5GED3Y9K2Q46JF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:39.819" grpc.time_ms=15.463 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 222.930872] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 59 vm-test-run-nixbot-gitlab> gitlab # [ 223.014473] gitaly[1382]: time="2026-06-29T22:08:39.921Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:09:09.655" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:39.655" pid=1382 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 223.015120] gitaly[1382]: time="2026-06-29T22:08:39.921Z" level=info msg="optimized repository" clean_stale_data_time=88.041916ms clean_work_trees_time="115.937µs" commit_graph_time="8.101µs" correlation_id=01KWAPTSS07AE46RFZBJM1KZEM gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.102µs" pid=1382 prune_time="7.823µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.939µs" storage_name=default system=housekeeping total_time=88.433586ms vm-test-run-nixbot-gitlab> gitlab # [ 223.015911] gitaly[1382]: time="2026-06-29T22:08:39.921Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T22:09:09.655" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T22:08:39.655" pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 223.060279] gitaly[1382]: time="2026-06-29T22:08:39.967Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T22:09:09.065" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:39.965" grpc.time_ms=1.352 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 223.065910] gitaly[1382]: time="2026-06-29T22:08:39.972Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTTF7G04T16REJR18VBBK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:39.963" pid=1382 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 223.067093] gitaly[1382]: time="2026-06-29T22:08:39.972Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTTF7G04T16REJR18VBBK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:39.963" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 223.067940] gitlab-workhorse[1282]: time="2026-06-29T22:08:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 223.069899] gitlab-workhorse[1282]: gitlab 2001:db8:1::1 - - [2026/06/29:22:08:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 62 vm-test-run-nixbot-gitlab> gitlab # [ 223.071134] gitaly[1382]: time="2026-06-29T22:08:39.973Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTTF7G04T16REJR18VBBK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T22:08:39.963" grpc.time_ms=9.543 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 223.087439] gitaly[1382]: time="2026-06-29T22:08:39.994Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:09.093" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:39.993" grpc.time_ms=0.678 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 223.160224] gitaly[1382]: time="2026-06-29T22:08:40.023Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAPTSS07AE46RFZBJM1KZEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T22:09:10.021" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T22:08:40.022" grpc.time_ms=0.739 pid=1382 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 223.161753] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.033Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"19d1038ff113ee6347af6d9e","created_at":"2026-06-29T22:08:39.443Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-e830e5306e1c40fb","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.446Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-19d1038ff113ee6347af6d9e: done: 0.492024 sec","job_status":"done","queue_duration_s":0.094322,"scheduling_latency_s":0.094363,"gitaly_calls":4,"gitaly_duration_s":0.265672,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.018215,"redis_read_bytes":173,"redis_write_bytes":20926,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001566,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002185,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.004336,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13379,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.005195,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000279,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001118,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.115933,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.420488917,"duration_s":0.492024,"completed_at":"2026-06-29T22:08:40.032Z","load_balancing_strategy":"primary","db_duration_s":0.050928,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 223.249054] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.034Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"0d4410b251169016c7a36989","created_at":"2026-06-29T22:08:39.648Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.650Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-0d4410b251169016c7a36989: start","job_status":"start","queue_duration_s":0.383726,"scheduling_latency_s":0.383749} vm-test-run-nixbot-gitlab> gitlab # [ 223.250317] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.063Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"0d4410b251169016c7a36989","created_at":"2026-06-29T22:08:39.648Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.650Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-0d4410b251169016c7a36989: done: 0.029506 sec","job_status":"done","queue_duration_s":0.383726,"scheduling_latency_s":0.383749,"redis_calls":7,"redis_duration_s":0.001685,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001244,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029506,"completed_at":"2026-06-29T22:08:40.063Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004892,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 223.320980] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.065Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1443dcd915b6370e8d3b201e","created_at":"2026-06-29T22:08:39.927Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.930Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1443dcd915b6370e8d3b201e: start","job_status":"start","queue_duration_s":0.135191,"scheduling_latency_s":0.135252} vm-test-run-nixbot-gitlab> gitlab # [ 223.322251] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.083Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1443dcd915b6370e8d3b201e","created_at":"2026-06-29T22:08:39.927Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.930Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1443dcd915b6370e8d3b201e: done: 0.018162 sec","job_status":"done","queue_duration_s":0.135191,"scheduling_latency_s":0.135252,"redis_calls":7,"redis_duration_s":0.001629,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018162,"completed_at":"2026-06-29T22:08:40.083Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.406353] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.085Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"edb356e0513ddd29001c6414","created_at":"2026-06-29T22:08:39.937Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.940Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-edb356e0513ddd29001c6414: start","job_status":"start","queue_duration_s":0.144037,"scheduling_latency_s":0.144057} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses vm-test-run-nixbot-gitlab> gitlab # [ 223.410197] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"edb356e0513ddd29001c6414","created_at":"2026-06-29T22:08:39.937Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.940Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-edb356e0513ddd29001c6414: done: 0.03708 sec","job_status":"done","queue_duration_s":0.144037,"scheduling_latency_s":0.144057,"redis_calls":7,"redis_duration_s":0.0017009999999999998,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001249,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03708,"completed_at":"2026-06-29T22:08:40.121Z","load_balancing_strategy":"primary","db_duration_s":0.018584,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.485197] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.123Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"18a3350e85d8c9716513e725","created_at":"2026-06-29T22:08:39.948Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.951Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-18a3350e85d8c9716513e725: start","job_status":"start","queue_duration_s":0.172365,"scheduling_latency_s":0.172389} vm-test-run-nixbot-gitlab> gitlab # [ 223.557364] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.145Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"18a3350e85d8c9716513e725","created_at":"2026-06-29T22:08:39.948Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.951Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-18a3350e85d8c9716513e725: done: 0.021949 sec","job_status":"done","queue_duration_s":0.172365,"scheduling_latency_s":0.172389,"redis_calls":10,"redis_duration_s":0.002408,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000711,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021949,"completed_at":"2026-06-29T22:08:40.145Z","load_balancing_strategy":"primary","db_duration_s":0.002895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.569004] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.146Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e75b9338d43b862e6590808b","created_at":"2026-06-29T22:08:39.951Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.953Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e75b9338d43b862e6590808b: start","job_status":"start","queue_duration_s":0.193701,"scheduling_latency_s":0.193722} vm-test-run-nixbot-gitlab> gitlab # [ 223.595868] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.164Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e75b9338d43b862e6590808b","created_at":"2026-06-29T22:08:39.951Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.953Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e75b9338d43b862e6590808b: done: 0.017984 sec","job_status":"done","queue_duration_s":0.193701,"scheduling_latency_s":0.193722,"redis_calls":7,"redis_duration_s":0.001633,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001191,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017984,"completed_at":"2026-06-29T22:08:40.164Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002867,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.681573] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.166Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0f4a8c71c9196f34fdcc7705","created_at":"2026-06-29T22:08:39.953Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.956Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0f4a8c71c9196f34fdcc7705: start","job_status":"start","queue_duration_s":0.210154,"scheduling_latency_s":0.210175} vm-test-run-nixbot-gitlab> gitlab # [ 223.688692] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0f4a8c71c9196f34fdcc7705","created_at":"2026-06-29T22:08:39.953Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.956Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0f4a8c71c9196f34fdcc7705: done: 0.019279 sec","job_status":"done","queue_duration_s":0.210154,"scheduling_latency_s":0.210175,"redis_calls":8,"redis_duration_s":0.001913,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001431,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019279,"completed_at":"2026-06-29T22:08:40.185Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002437,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/c0de400263dc825e5f844f505cbeaf8de2ec47cd/statuses, in 0.31 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T22:08:29.920Z', 'started_at': None, 'finished_at': '2026-06-29T22:08:35.077Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'c0de400263dc825e5f844f505cbeaf8de2ec47cd', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-06-29T22:08:35.231Z', 'started_at': None, 'finished_at': '2026-06-29T22:08:39.435Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 10.58 seconds) vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 261.84 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 223.757089] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.186Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e61440f6a3e4155a9febfac4","created_at":"2026-06-29T22:08:39.956Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.958Z","job_size_bytes":66,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e61440f6a3e4155a9febfac4: start","job_status":"start","queue_duration_s":0.228742,"scheduling_latency_s":0.228764} vm-test-run-nixbot-gitlab> gitlab # [ 223.849912] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.202Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e61440f6a3e4155a9febfac4","created_at":"2026-06-29T22:08:39.956Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T22:08:39.958Z","job_size_bytes":66,"pid":2114,"sidekiq_tid":"mabi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e61440f6a3e4155a9febfac4: done: 0.015721 sec","job_status":"done","queue_duration_s":0.228742,"scheduling_latency_s":0.228764,"redis_calls":7,"redis_duration_s":0.001981,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001542,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015721,"completed_at":"2026-06-29T22:08:40.202Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001181,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> test script finished in 261.95s vm-test-run-nixbot-gitlab> cleanup vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13) vm-test-run-nixbot-gitlab> gitlab # [ 223.924328] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T22:08:40.204Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"62d0ca78b27dd424ef2e3414","created_at":"2026-06-29T22:08:39.959Z","trace_propagation_headers":{"sentry-trace":"2bc6e7f45edf40f6ae6d82722cd639cb-16394b25995f4196","baggage":"sentry-trace_id=2bc6e7f45edf40f6ae6d82722cd639cb,sentry-release=Unknown"},"correlation_id":"01KWAPTSS07AE46RFZBJM1KZEM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primaryqemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13) vm-test-run-nixbot-gitlab> gitlab # " vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.33 seconds) post-build step Upload coverage to codecov: ok Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab