1these 13 derivations will be built:2 /nix/store/5m8m5liqi0rzw3ys08zvpcw71k6q6cqv-python3.13-nixbot.drv3 /nix/store/xrh8zxpwr9az5yvgp3v7i6qjdxbhp33r-unit-nixbot.service.drv4 /nix/store/cj56prmrrkkj4z3myji5pqi6w5a3nxqz-system-units.drv5 /nix/store/sd2l4n51rgq82ix0ad97ayz8c4cf6k8f-etc.drv6 /nix/store/l4ax29l3yl9r3mbnyf9gxf3kwb5029lq-activate.drv7 /nix/store/z6jvh32zvdqb99fbjg083clf2lhhmngh-nixos-system-gitlab-test.drv8 /nix/store/xb0p2fq6iq43blqqprbwnv1x7zggk2kn-closure-info.drv9 /nix/store/kyiizkmn392mimk38dmm7s8zi4m6691l-closure-info.drv10 /nix/store/vlx0175j29sjibg0g0fwr84j0sh3gd3z-run-nixos-vm.drv11 /nix/store/100fczdjywm9hzacf4b04hc9qvprhkxk-nixos-vm.drv12 /nix/store/r9hchrrmmx95x13vgf9k19gdi729l8b5-driverConfiguration.json.drv13 /nix/store/s2hvdax83f9ndcs6d8adlbvgambsdab8-nixos-test-driver-nixbot-gitlab.drv14 /nix/store/grdan0lr1zlbkjr6c313j34ix6flrpx7-vm-test-run-nixbot-gitlab.drv15building '/nix/store/5m8m5liqi0rzw3ys08zvpcw71k6q6cqv-python3.13-nixbot.drv' on 'ssh-ng://nix@jamie'16building '/nix/store/5m8m5liqi0rzw3ys08zvpcw71k6q6cqv-python3.13-nixbot.drv'17python3.13-nixbot> Sourcing python-remove-tests-dir-hook18python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh19python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh20python3.13-nixbot> Sourcing pypa-build-hook21python3.13-nixbot> Using pypaBuildPhase22python3.13-nixbot> Sourcing python-runtime-deps-check-hook23python3.13-nixbot> Using pythonRuntimeDepsCheckHook24python3.13-nixbot> Sourcing pypa-install-hook25python3.13-nixbot> Using pypaInstallPhase26python3.13-nixbot> Sourcing python-imports-check-hook.sh27python3.13-nixbot> Using pythonImportsCheckPhase28python3.13-nixbot> Sourcing python-namespaces-hook29python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh30python3.13-nixbot> Running phase: unpackPhase31python3.13-nixbot> unpacking source archive /nix/store/lx5k613j9may44lv7cpbvqj1cz13xgg8-nixbot32python3.13-nixbot> source root is nixbot33python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml"34python3.13-nixbot> Running phase: patchPhase35python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase36python3.13-nixbot> Running phase: configurePhase37python3.13-nixbot> no configure script, doing nothing38python3.13-nixbot> Running phase: buildPhase39python3.13-nixbot> Executing pypaBuildPhase40python3.13-nixbot> Creating a wheel...41python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel42python3.13-nixbot> * Getting build dependencies for wheel...43python3.13-nixbot> * Building wheel...44python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl45python3.13-nixbot> Finished creating a wheel...46python3.13-nixbot> Finished executing pypaBuildPhase47python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook48python3.13-nixbot> Executing pythonRuntimeDepsCheck49python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl50python3.13-nixbot> Finished executing pythonRuntimeDepsCheck51python3.13-nixbot> Running phase: installPhase52python3.13-nixbot> Executing pypaInstallPhase53python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl54python3.13-nixbot> Finished executing pypaInstallPhase55python3.13-nixbot> Running phase: pythonOutputDistPhase56python3.13-nixbot> Executing pythonOutputDistPhase57python3.13-nixbot> Finished executing pythonOutputDistPhase58python3.13-nixbot> Running phase: fixupPhase59python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/73r4iypmg1ia100q99r4i5q9s3sz114g-python3.13-nixbot60python3.13-nixbot> checking for references to /build/ in /nix/store/73r4iypmg1ia100q99r4i5q9s3sz114g-python3.13-nixbot...61python3.13-nixbot> patching script interpreter paths in /nix/store/73r4iypmg1ia100q99r4i5q9s3sz114g-python3.13-nixbot62python3.13-nixbot> stripping (with command strip and flags -S -p) in /nix/store/73r4iypmg1ia100q99r4i5q9s3sz114g-python3.13-nixbot/lib /nix/store/73r4iypmg1ia100q99r4i5q9s3sz114g-python3.13-nixbot/bin63python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/gw0apy0lrigkwb86fva2bz4jhjq31zhq-python3.13-nixbot-dist64python3.13-nixbot> checking for references to /build/ in /nix/store/gw0apy0lrigkwb86fva2bz4jhjq31zhq-python3.13-nixbot-dist...65python3.13-nixbot> patching script interpreter paths in /nix/store/gw0apy0lrigkwb86fva2bz4jhjq31zhq-python3.13-nixbot-dist66python3.13-nixbot> Rewriting #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13 to #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.1367python3.13-nixbot> wrapping `/nix/store/73r4iypmg1ia100q99r4i5q9s3sz114g-python3.13-nixbot/bin/nixbot'...68python3.13-nixbot> Executing pythonRemoveTestsDir69python3.13-nixbot> Finished executing pythonRemoveTestsDir70python3.13-nixbot> Running phase: pythonCatchConflictsPhase71python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase72python3.13-nixbot> Running phase: pythonImportsCheckPhase73python3.13-nixbot> Executing pythonImportsCheckPhase74building '/nix/store/xrh8zxpwr9az5yvgp3v7i6qjdxbhp33r-unit-nixbot.service.drv'75building '/nix/store/cj56prmrrkkj4z3myji5pqi6w5a3nxqz-system-units.drv'76building '/nix/store/sd2l4n51rgq82ix0ad97ayz8c4cf6k8f-etc.drv'77building '/nix/store/l4ax29l3yl9r3mbnyf9gxf3kwb5029lq-activate.drv'78building '/nix/store/z6jvh32zvdqb99fbjg083clf2lhhmngh-nixos-system-gitlab-test.drv'79nixos-system-gitlab-test> structuredAttrs is enabled80building '/nix/store/xb0p2fq6iq43blqqprbwnv1x7zggk2kn-closure-info.drv'81closure-info> structuredAttrs is enabled82building '/nix/store/kyiizkmn392mimk38dmm7s8zi4m6691l-closure-info.drv'83closure-info> structuredAttrs is enabled84building '/nix/store/vlx0175j29sjibg0g0fwr84j0sh3gd3z-run-nixos-vm.drv'85building '/nix/store/100fczdjywm9hzacf4b04hc9qvprhkxk-nixos-vm.drv'86building '/nix/store/r9hchrrmmx95x13vgf9k19gdi729l8b5-driverConfiguration.json.drv'87driverConfiguration.json> structuredAttrs is enabled88building '/nix/store/s2hvdax83f9ndcs6d8adlbvgambsdab8-nixos-test-driver-nixbot-gitlab.drv'89nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)90nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck91nixos-test-driver-nixbot-gitlab> All checks passed!92nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)93nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint94nixos-test-driver-nixbot-gitlab> All checks passed!95building '/nix/store/grdan0lr1zlbkjr6c313j34ix6flrpx7-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie'96building '/nix/store/grdan0lr1zlbkjr6c313j34ix6flrpx7-vm-test-run-nixbot-gitlab.drv'97vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state98vm-test-run-nixbot-gitlab> start all VLans99vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)100vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds101vm-test-run-nixbot-gitlab> run the VM test script102vm-test-run-nixbot-gitlab> additionally exposed symbols:103vm-test-run-nixbot-gitlab> gitlab,104vm-test-run-nixbot-gitlab> vlan1,105vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh106vm-test-run-nixbot-gitlab> start all VMs107vm-test-run-nixbot-gitlab> gitlab: starting vm108vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)109vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...110vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.CcqT6APYSO', fmt=raw size=1073741824111vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)112vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done113vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes114vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: bbd9d481-6050-4ef5-a9fc-6069af9bfffc115vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:116vm-test-run-nixbot-gitlab> (finished: start all VMs, in 34.52 seconds)117vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376118vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service119vm-test-run-nixbot-gitlab> gitlab # 120vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting121vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done122vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done123vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done124vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done125vm-test-run-nixbot-gitlab> gitlab # 126vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.127vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...128vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.129vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)130vm-test-run-nixbot-gitlab> gitlab # 131vm-test-run-nixbot-gitlab> gitlab # 132vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00133vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...134vm-test-run-nixbot-gitlab> gitlab # 135vm-test-run-nixbot-gitlab> gitlab # 136vm-test-run-nixbot-gitlab> gitlab # 137vm-test-run-nixbot-gitlab> gitlab # 138vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00139vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...140vm-test-run-nixbot-gitlab> gitlab # 141vm-test-run-nixbot-gitlab> gitlab # 142vm-test-run-nixbot-gitlab> gitlab # Booting from ROM...143vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok144vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026145vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qibpq3z8311i99pbi450wlcmbny123z4-nixos-system-gitlab-test/init regInfo=/nix/store/vgcvhbfqcfi2ym2ny0bd9p124s2bclm6-closure-info/registration console=ttyS0,115200n8 console=tty0146vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-provided physical RAM map:147vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable148vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved149vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved150vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable151vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved152vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved153vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved154vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable155vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved156vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NX (Execute Disable) protection: active157vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] APIC: Static calls initialized158vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SMBIOS 2.8 present.159vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014160vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: Memory slots populated: 1/1161vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Hypervisor detected: KVM162vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000163vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00164vm-test-run-nixbot-gitlab> gitlab # [ 0.000001] kvm-clock: using sched offset of 482932927 cycles165vm-test-run-nixbot-gitlab> gitlab # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns166vm-test-run-nixbot-gitlab> gitlab # [ 0.000006] tsc: Detected 2400.010 MHz processor167vm-test-run-nixbot-gitlab> gitlab # [ 0.000815] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000168vm-test-run-nixbot-gitlab> gitlab # [ 0.000851] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs169vm-test-run-nixbot-gitlab> gitlab # [ 0.000854] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT170vm-test-run-nixbot-gitlab> gitlab # [ 0.000907] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000171vm-test-run-nixbot-gitlab> gitlab # [ 0.002761] found SMP MP-table at [mem 0x000f5470-0x000f547f]172vm-test-run-nixbot-gitlab> gitlab # [ 0.002771] Using GB pages for direct mapping173vm-test-run-nixbot-gitlab> gitlab # [ 0.002900] RAMDISK: [mem 0xbe4e0000-0xbffcffff]174vm-test-run-nixbot-gitlab> gitlab # [ 0.002908] ACPI: Early table checksum verification disabled175vm-test-run-nixbot-gitlab> gitlab # [ 0.002911] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )176vm-test-run-nixbot-gitlab> gitlab # [ 0.002915] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)177vm-test-run-nixbot-gitlab> gitlab # [ 0.002919] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)178vm-test-run-nixbot-gitlab> gitlab # [ 0.002927] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)179vm-test-run-nixbot-gitlab> gitlab # [ 0.002929] ACPI: FACS 0x00000000BFFE0000 000040180vm-test-run-nixbot-gitlab> gitlab # [ 0.002930] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)181vm-test-run-nixbot-gitlab> gitlab # [ 0.002932] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)182vm-test-run-nixbot-gitlab> gitlab # [ 0.002933] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)183vm-test-run-nixbot-gitlab> gitlab # [ 0.002934] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]184vm-test-run-nixbot-gitlab> gitlab # [ 0.002936] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]185vm-test-run-nixbot-gitlab> gitlab # [ 0.002936] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]186vm-test-run-nixbot-gitlab> gitlab # [ 0.002937] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]187vm-test-run-nixbot-gitlab> gitlab # [ 0.002937] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]188vm-test-run-nixbot-gitlab> gitlab # [ 0.002938] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]189vm-test-run-nixbot-gitlab> gitlab # [ 0.003434] No NUMA configuration found190vm-test-run-nixbot-gitlab> gitlab # [ 0.003436] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]191vm-test-run-nixbot-gitlab> gitlab # [ 0.003439] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]192vm-test-run-nixbot-gitlab> gitlab # [ 0.006234] Zone ranges:193vm-test-run-nixbot-gitlab> gitlab # [ 0.006235] DMA [mem 0x0000000000001000-0x0000000000ffffff]194vm-test-run-nixbot-gitlab> gitlab # [ 0.006237] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]195vm-test-run-nixbot-gitlab> gitlab # [ 0.006238] Normal [mem 0x0000000100000000-0x00000001bfffffff]196vm-test-run-nixbot-gitlab> gitlab # [ 0.006239] Device empty197vm-test-run-nixbot-gitlab> gitlab # [ 0.006240] Movable zone start for each node198vm-test-run-nixbot-gitlab> gitlab # [ 0.006241] Early memory node ranges199vm-test-run-nixbot-gitlab> gitlab # [ 0.006241] node 0: [mem 0x0000000000001000-0x000000000009efff]200vm-test-run-nixbot-gitlab> gitlab # [ 0.006242] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]201vm-test-run-nixbot-gitlab> gitlab # [ 0.006243] node 0: [mem 0x0000000100000000-0x00000001bfffffff]202vm-test-run-nixbot-gitlab> gitlab # [ 0.006244] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]203vm-test-run-nixbot-gitlab> gitlab # [ 0.006264] On node 0, zone DMA: 1 pages in unavailable ranges204vm-test-run-nixbot-gitlab> gitlab # [ 0.006544] On node 0, zone DMA: 97 pages in unavailable ranges205vm-test-run-nixbot-gitlab> gitlab # [ 0.119563] On node 0, zone Normal: 40 pages in unavailable ranges206vm-test-run-nixbot-gitlab> gitlab # [ 0.120586] ACPI: PM-Timer IO Port: 0x608207vm-test-run-nixbot-gitlab> gitlab # [ 0.120604] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])208vm-test-run-nixbot-gitlab> gitlab # [ 0.120640] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23209vm-test-run-nixbot-gitlab> gitlab # [ 0.120643] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)210vm-test-run-nixbot-gitlab> gitlab # [ 0.120644] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)211vm-test-run-nixbot-gitlab> gitlab # [ 0.120646] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)212vm-test-run-nixbot-gitlab> gitlab # [ 0.120647] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)213vm-test-run-nixbot-gitlab> gitlab # [ 0.120648] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)214vm-test-run-nixbot-gitlab> gitlab # [ 0.120650] ACPI: Using ACPI (MADT) for SMP configuration information215vm-test-run-nixbot-gitlab> gitlab # [ 0.120651] ACPI: HPET id: 0x8086a201 base: 0xfed00000216vm-test-run-nixbot-gitlab> gitlab # [ 0.120658] TSC deadline timer available217vm-test-run-nixbot-gitlab> gitlab # [ 0.120662] CPU topo: Max. logical packages: 1218vm-test-run-nixbot-gitlab> gitlab # [ 0.120663] CPU topo: Max. logical dies: 1219vm-test-run-nixbot-gitlab> gitlab # [ 0.120663] CPU topo: Max. dies per package: 1220vm-test-run-nixbot-gitlab> gitlab # [ 0.120667] CPU topo: Max. threads per core: 1221vm-test-run-nixbot-gitlab> gitlab # [ 0.120667] CPU topo: Num. cores per package: 4222vm-test-run-nixbot-gitlab> gitlab # [ 0.120668] CPU topo: Num. threads per package: 4223vm-test-run-nixbot-gitlab> gitlab # [ 0.120668] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs224vm-test-run-nixbot-gitlab> gitlab # [ 0.120695] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()225vm-test-run-nixbot-gitlab> gitlab # [ 0.120714] kvm-guest: KVM setup pv remote TLB flush226vm-test-run-nixbot-gitlab> gitlab # [ 0.120717] kvm-guest: setup PV sched yield227vm-test-run-nixbot-gitlab> gitlab # [ 0.120738] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]228vm-test-run-nixbot-gitlab> gitlab # [ 0.120740] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]229vm-test-run-nixbot-gitlab> gitlab # [ 0.120741] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]230vm-test-run-nixbot-gitlab> gitlab # [ 0.120742] [mem 0xc0000000-0xfeffbfff] available for PCI devices231vm-test-run-nixbot-gitlab> gitlab # [ 0.120744] Booting paravirtualized kernel on KVM232vm-test-run-nixbot-gitlab> gitlab # [ 0.120748] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns233vm-test-run-nixbot-gitlab> gitlab # [ 0.125214] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1234vm-test-run-nixbot-gitlab> gitlab # [ 0.127545] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288235vm-test-run-nixbot-gitlab> gitlab # [ 0.127591] kvm-guest: PV spinlocks enabled236vm-test-run-nixbot-gitlab> gitlab # [ 0.127593] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)237vm-test-run-nixbot-gitlab> gitlab # [ 0.127596] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qibpq3z8311i99pbi450wlcmbny123z4-nixos-system-gitlab-test/init regInfo=/nix/store/vgcvhbfqcfi2ym2ny0bd9p124s2bclm6-closure-info/registration console=ttyS0,115200n8 console=tty0238vm-test-run-nixbot-gitlab> gitlab # [ 0.127694] Unknown kernel command line parameters "regInfo=/nix/store/vgcvhbfqcfi2ym2ny0bd9p124s2bclm6-closure-info/registration", will be passed to user space.239vm-test-run-nixbot-gitlab> gitlab # [ 0.127707] random: crng init done240vm-test-run-nixbot-gitlab> gitlab # [ 0.127707] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes241vm-test-run-nixbot-gitlab> gitlab # [ 0.136988] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)242vm-test-run-nixbot-gitlab> gitlab # [ 0.141783] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)243vm-test-run-nixbot-gitlab> gitlab # [ 0.141860] software IO TLB: area num 4.244vm-test-run-nixbot-gitlab> gitlab # [ 0.220256] Fallback order for Node 0: 0245vm-test-run-nixbot-gitlab> gitlab # [ 0.220265] Built 1 zonelists, mobility grouping on. Total pages: 1572726246vm-test-run-nixbot-gitlab> gitlab # [ 0.220267] Policy zone: Normal247vm-test-run-nixbot-gitlab> gitlab # [ 0.222938] mem auto-init: stack:all(zero), heap alloc:on, heap free:off248vm-test-run-nixbot-gitlab> gitlab # [ 0.232554] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1249vm-test-run-nixbot-gitlab> gitlab # [ 0.247138] allocated 12582912 bytes of page_ext250vm-test-run-nixbot-gitlab> gitlab # [ 0.257116] ftrace: allocating 48565 entries in 192 pages251vm-test-run-nixbot-gitlab> gitlab # [ 0.257118] ftrace: allocated 192 pages with 2 groups252vm-test-run-nixbot-gitlab> gitlab # [ 0.258027] Dynamic Preempt: lazy253vm-test-run-nixbot-gitlab> gitlab # [ 0.258192] rcu: Preemptible hierarchical RCU implementation.254vm-test-run-nixbot-gitlab> gitlab # [ 0.258193] rcu: RCU event tracing is enabled.255vm-test-run-nixbot-gitlab> gitlab # [ 0.258194] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.256vm-test-run-nixbot-gitlab> gitlab # [ 0.258195] Trampoline variant of Tasks RCU enabled.257vm-test-run-nixbot-gitlab> gitlab # [ 0.258196] Rude variant of Tasks RCU enabled.258vm-test-run-nixbot-gitlab> gitlab # [ 0.258196] Tracing variant of Tasks RCU enabled.259vm-test-run-nixbot-gitlab> gitlab # [ 0.258197] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.260vm-test-run-nixbot-gitlab> gitlab # [ 0.258198] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4261vm-test-run-nixbot-gitlab> gitlab # [ 0.258224] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.262vm-test-run-nixbot-gitlab> gitlab # [ 0.258226] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.263vm-test-run-nixbot-gitlab> gitlab # [ 0.258227] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.264vm-test-run-nixbot-gitlab> gitlab # [ 0.262585] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16265vm-test-run-nixbot-gitlab> gitlab # [ 0.262868] rcu: srcu_init: Setting srcu_struct sizes based on contention.266vm-test-run-nixbot-gitlab> gitlab # [ 0.262988] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)267vm-test-run-nixbot-gitlab> gitlab # [ 0.270266] Console: colour VGA+ 80x25268vm-test-run-nixbot-gitlab> gitlab # [ 0.270270] printk: legacy console [tty0] enabled269vm-test-run-nixbot-gitlab> gitlab # [ 0.314441] printk: legacy console [ttyS0] enabled270vm-test-run-nixbot-gitlab> gitlab # [ 0.515409] ACPI: Core revision 20250807271vm-test-run-nixbot-gitlab> gitlab # [ 0.517014] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns272vm-test-run-nixbot-gitlab> gitlab # [ 0.519951] APIC: Switch to symmetric I/O mode setup273vm-test-run-nixbot-gitlab> gitlab # [ 0.521748] x2apic enabled274vm-test-run-nixbot-gitlab> gitlab # [ 0.523004] APIC: Switched APIC routing to: physical x2apic275vm-test-run-nixbot-gitlab> gitlab # [ 0.524751] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()276vm-test-run-nixbot-gitlab> gitlab # [ 0.527051] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()277vm-test-run-nixbot-gitlab> gitlab # [ 0.529849] kvm-guest: setup PV IPIs278vm-test-run-nixbot-gitlab> gitlab # [ 0.532247] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1279vm-test-run-nixbot-gitlab> gitlab # [ 0.534152] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns280vm-test-run-nixbot-gitlab> gitlab # [ 0.537295] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)281vm-test-run-nixbot-gitlab> gitlab # [ 0.539309] x86/cpu: User Mode Instruction Prevention (UMIP) activated282vm-test-run-nixbot-gitlab> gitlab # [ 0.540451] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127283vm-test-run-nixbot-gitlab> gitlab # [ 0.541294] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0284vm-test-run-nixbot-gitlab> gitlab # [ 0.542298] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto285vm-test-run-nixbot-gitlab> gitlab # [ 0.544293] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl286vm-test-run-nixbot-gitlab> gitlab # [ 0.546294] Transient Scheduler Attacks: Mitigation: Clear CPU buffers287vm-test-run-nixbot-gitlab> gitlab # [ 0.547293] Spectre V2 : Mitigation: Enhanced / Automatic IBRS288vm-test-run-nixbot-gitlab> gitlab # [ 0.548293] Speculative Return Stack Overflow: Mitigation: Safe RET289vm-test-run-nixbot-gitlab> gitlab # [ 0.550292] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization290vm-test-run-nixbot-gitlab> gitlab # [ 0.551299] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier291vm-test-run-nixbot-gitlab> gitlab # [ 0.553294] active return thunk: srso_alias_return_thunk292vm-test-run-nixbot-gitlab> gitlab # [ 0.554322] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'293vm-test-run-nixbot-gitlab> gitlab # [ 0.556293] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'294vm-test-run-nixbot-gitlab> gitlab # [ 0.557293] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'295vm-test-run-nixbot-gitlab> gitlab # [ 0.558293] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'296vm-test-run-nixbot-gitlab> gitlab # [ 0.560293] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'297vm-test-run-nixbot-gitlab> gitlab # [ 0.562293] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'298vm-test-run-nixbot-gitlab> gitlab # [ 0.564293] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'299vm-test-run-nixbot-gitlab> gitlab # [ 0.566293] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'300vm-test-run-nixbot-gitlab> gitlab # [ 0.568293] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'301vm-test-run-nixbot-gitlab> gitlab # [ 0.569293] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256302vm-test-run-nixbot-gitlab> gitlab # [ 0.571293] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64303vm-test-run-nixbot-gitlab> gitlab # [ 0.572293] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512304vm-test-run-nixbot-gitlab> gitlab # [ 0.573293] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024305vm-test-run-nixbot-gitlab> gitlab # [ 0.574293] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8306vm-test-run-nixbot-gitlab> gitlab # [ 0.575293] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16307vm-test-run-nixbot-gitlab> gitlab # [ 0.577293] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24308vm-test-run-nixbot-gitlab> gitlab # [ 0.579293] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.309vm-test-run-nixbot-gitlab> gitlab # [ 0.609764] Freeing SMP alternatives memory: 44K310vm-test-run-nixbot-gitlab> gitlab # [ 0.610296] pid_max: default: 32768 minimum: 301311vm-test-run-nixbot-gitlab> gitlab # [ 0.612429] LSM: initializing lsm=capability,landlock,yama,bpf,ima312vm-test-run-nixbot-gitlab> gitlab # [ 0.613422] landlock: Up and running.313vm-test-run-nixbot-gitlab> gitlab # [ 0.614293] Yama: becoming mindful.314vm-test-run-nixbot-gitlab> gitlab # [ 0.615502] LSM support for eBPF active315vm-test-run-nixbot-gitlab> gitlab # [ 0.617620] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)316vm-test-run-nixbot-gitlab> gitlab # [ 0.618449] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)317vm-test-run-nixbot-gitlab> gitlab # [ 0.621818] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)318vm-test-run-nixbot-gitlab> gitlab # [ 0.623017] Performance Events: Fam17h+ core perfctr, AMD PMU driver.319vm-test-run-nixbot-gitlab> gitlab # [ 0.624297] ... version: 2320vm-test-run-nixbot-gitlab> gitlab # [ 0.625303] ... bit width: 48321vm-test-run-nixbot-gitlab> gitlab # [ 0.626295] ... generic counters: 6322vm-test-run-nixbot-gitlab> gitlab # [ 0.627295] ... generic bitmap: 000000000000003f323vm-test-run-nixbot-gitlab> gitlab # [ 0.628295] ... fixed-purpose counters: 0324vm-test-run-nixbot-gitlab> gitlab # [ 0.629295] ... fixed-purpose bitmap: 0000000000000000325vm-test-run-nixbot-gitlab> gitlab # [ 0.630307] ... value mask: 0000ffffffffffff326vm-test-run-nixbot-gitlab> gitlab # [ 0.631295] ... max period: 00007fffffffffff327vm-test-run-nixbot-gitlab> gitlab # [ 0.632295] ... global_ctrl mask: 000000000000003f328vm-test-run-nixbot-gitlab> gitlab # [ 0.633464] signal: max sigframe size: 3376329vm-test-run-nixbot-gitlab> gitlab # [ 0.634371] rcu: Hierarchical SRCU implementation.330vm-test-run-nixbot-gitlab> gitlab # [ 0.635300] rcu: Max phase no-delay instances is 400.331vm-test-run-nixbot-gitlab> gitlab # [ 0.636499] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level332vm-test-run-nixbot-gitlab> gitlab # [ 0.641871] smp: Bringing up secondary CPUs ...333vm-test-run-nixbot-gitlab> gitlab # [ 0.642729] smpboot: x86: Booting SMP configuration:334vm-test-run-nixbot-gitlab> gitlab # [ 0.643298] .... node #0, CPUs: #1 #2 #3335vm-test-run-nixbot-gitlab> gitlab # [ 0.647397] smp: Brought up 1 node, 4 CPUs336vm-test-run-nixbot-gitlab> gitlab # [ 0.650298] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)337vm-test-run-nixbot-gitlab> gitlab # [ 0.651893] Memory: 6013460K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258300K reserved, 0K cma-reserved)338vm-test-run-nixbot-gitlab> gitlab # [ 0.654294] devtmpfs: initialized339vm-test-run-nixbot-gitlab> gitlab # [ 0.655622] x86/mm: Memory block size: 128MB340vm-test-run-nixbot-gitlab> gitlab # [ 0.659303] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns341vm-test-run-nixbot-gitlab> gitlab # [ 0.660345] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)342vm-test-run-nixbot-gitlab> gitlab # [ 0.661388] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).343vm-test-run-nixbot-gitlab> gitlab # [ 0.662416] pinctrl core: initialized pinctrl subsystem344vm-test-run-nixbot-gitlab> gitlab # [ 0.663885] PM: RTC time: 14:51:25, date: 2026-06-17345vm-test-run-nixbot-gitlab> gitlab # [ 0.667430] NET: Registered PF_NETLINK/PF_ROUTE protocol family346vm-test-run-nixbot-gitlab> gitlab # [ 0.669812] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations347vm-test-run-nixbot-gitlab> gitlab # [ 0.670363] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations348vm-test-run-nixbot-gitlab> gitlab # [ 0.672829] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations349vm-test-run-nixbot-gitlab> gitlab # [ 0.673310] audit: initializing netlink subsys (disabled)350vm-test-run-nixbot-gitlab> gitlab # [ 0.674526] audit: type=2000 audit(1781707884.991:1): state=initialized audit_enabled=0 res=1351vm-test-run-nixbot-gitlab> gitlab # [ 0.674555] thermal_sys: Registered thermal governor 'fair_share'352vm-test-run-nixbot-gitlab> gitlab # [ 0.676295] thermal_sys: Registered thermal governor 'bang_bang'353vm-test-run-nixbot-gitlab> gitlab # [ 0.677295] thermal_sys: Registered thermal governor 'step_wise'354vm-test-run-nixbot-gitlab> gitlab # [ 0.678295] thermal_sys: Registered thermal governor 'user_space'355vm-test-run-nixbot-gitlab> gitlab # [ 0.679295] thermal_sys: Registered thermal governor 'power_allocator'356vm-test-run-nixbot-gitlab> gitlab # [ 0.680352] cpuidle: using governor menu357vm-test-run-nixbot-gitlab> gitlab # [ 0.683525] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5358vm-test-run-nixbot-gitlab> gitlab # [ 0.684640] PCI: Using configuration type 1 for base access359vm-test-run-nixbot-gitlab> gitlab # [ 0.685296] PCI: Using configuration type 1 for extended access360vm-test-run-nixbot-gitlab> gitlab # [ 0.686513] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.361vm-test-run-nixbot-gitlab> gitlab # [ 0.689353] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages362vm-test-run-nixbot-gitlab> gitlab # [ 0.690295] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page363vm-test-run-nixbot-gitlab> gitlab # [ 0.691295] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages364vm-test-run-nixbot-gitlab> gitlab # [ 0.692295] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page365vm-test-run-nixbot-gitlab> gitlab # [ 0.696973] ACPI: Added _OSI(Module Device)366vm-test-run-nixbot-gitlab> gitlab # [ 0.700303] ACPI: Added _OSI(Processor Device)367vm-test-run-nixbot-gitlab> gitlab # [ 0.703304] ACPI: Added _OSI(Processor Aggregator Device)368vm-test-run-nixbot-gitlab> gitlab # [ 0.712216] ACPI: 1 ACPI AML tables successfully acquired and loaded369vm-test-run-nixbot-gitlab> gitlab # [ 0.721561] ACPI: Interpreter enabled370vm-test-run-nixbot-gitlab> gitlab # [ 0.722343] ACPI: PM: (supports S0 S3 S4 S5)371vm-test-run-nixbot-gitlab> gitlab # [ 0.723303] ACPI: Using IOAPIC for interrupt routing372vm-test-run-nixbot-gitlab> gitlab # [ 0.725561] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug373vm-test-run-nixbot-gitlab> gitlab # [ 0.731301] PCI: Using E820 reservations for host bridge windows374vm-test-run-nixbot-gitlab> gitlab # [ 0.734848] ACPI: Enabled 2 GPEs in block 00 to 0F375vm-test-run-nixbot-gitlab> gitlab # [ 0.748785] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])376vm-test-run-nixbot-gitlab> gitlab # [ 0.752310] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]377vm-test-run-nixbot-gitlab> gitlab # [ 0.758559] acpiphp: Slot [3] registered378vm-test-run-nixbot-gitlab> gitlab # [ 0.761419] acpiphp: Slot [4] registered379vm-test-run-nixbot-gitlab> gitlab # [ 0.763420] acpiphp: Slot [5] registered380vm-test-run-nixbot-gitlab> gitlab # [ 0.766468] acpiphp: Slot [6] registered381vm-test-run-nixbot-gitlab> gitlab # [ 0.767356] acpiphp: Slot [7] registered382vm-test-run-nixbot-gitlab> gitlab # [ 0.769365] acpiphp: Slot [8] registered383vm-test-run-nixbot-gitlab> gitlab # [ 0.770370] acpiphp: Slot [9] registered384vm-test-run-nixbot-gitlab> gitlab # [ 0.771378] acpiphp: Slot [10] registered385vm-test-run-nixbot-gitlab> gitlab # [ 0.772366] acpiphp: Slot [11] registered386vm-test-run-nixbot-gitlab> gitlab # [ 0.774364] acpiphp: Slot [12] registered387vm-test-run-nixbot-gitlab> gitlab # [ 0.775356] acpiphp: Slot [13] registered388vm-test-run-nixbot-gitlab> gitlab # [ 0.776344] acpiphp: Slot [14] registered389vm-test-run-nixbot-gitlab> gitlab # [ 0.778329] acpiphp: Slot [15] registered390vm-test-run-nixbot-gitlab> gitlab # [ 0.779329] acpiphp: Slot [16] registered391vm-test-run-nixbot-gitlab> gitlab # [ 0.780347] acpiphp: Slot [17] registered392vm-test-run-nixbot-gitlab> gitlab # [ 0.781329] acpiphp: Slot [18] registered393vm-test-run-nixbot-gitlab> gitlab # [ 0.783337] acpiphp: Slot [19] registered394vm-test-run-nixbot-gitlab> gitlab # [ 0.784328] acpiphp: Slot [20] registered395vm-test-run-nixbot-gitlab> gitlab # [ 0.785349] acpiphp: Slot [21] registered396vm-test-run-nixbot-gitlab> gitlab # [ 0.787329] acpiphp: Slot [22] registered397vm-test-run-nixbot-gitlab> gitlab # [ 0.788478] acpiphp: Slot [23] registered398vm-test-run-nixbot-gitlab> gitlab # [ 0.791416] acpiphp: Slot [24] registered399vm-test-run-nixbot-gitlab> gitlab # [ 0.794385] acpiphp: Slot [25] registered400vm-test-run-nixbot-gitlab> gitlab # [ 0.798401] acpiphp: Slot [26] registered401vm-test-run-nixbot-gitlab> gitlab # [ 0.801383] acpiphp: Slot [27] registered402vm-test-run-nixbot-gitlab> gitlab # [ 0.804415] acpiphp: Slot [28] registered403vm-test-run-nixbot-gitlab> gitlab # [ 0.807390] acpiphp: Slot [29] registered404vm-test-run-nixbot-gitlab> gitlab # [ 0.810387] acpiphp: Slot [30] registered405vm-test-run-nixbot-gitlab> gitlab # [ 0.814423] acpiphp: Slot [31] registered406vm-test-run-nixbot-gitlab> gitlab # [ 0.817501] PCI host bridge to bus 0000:00407vm-test-run-nixbot-gitlab> gitlab # [ 0.820322] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]408vm-test-run-nixbot-gitlab> gitlab # [ 0.825306] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]409vm-test-run-nixbot-gitlab> gitlab # [ 0.830305] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]410vm-test-run-nixbot-gitlab> gitlab # [ 0.835337] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]411vm-test-run-nixbot-gitlab> gitlab # [ 0.840302] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]412vm-test-run-nixbot-gitlab> gitlab # [ 0.844303] pci_bus 0000:00: root bus resource [bus 00-ff]413vm-test-run-nixbot-gitlab> gitlab # [ 0.848213] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint414vm-test-run-nixbot-gitlab> gitlab # [ 0.854005] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint415vm-test-run-nixbot-gitlab> gitlab # [ 0.858279] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint416vm-test-run-nixbot-gitlab> gitlab # [ 0.863362] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]417vm-test-run-nixbot-gitlab> gitlab # [ 0.864361] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk418vm-test-run-nixbot-gitlab> gitlab # [ 0.866296] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk419vm-test-run-nixbot-gitlab> gitlab # [ 0.868296] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk420vm-test-run-nixbot-gitlab> gitlab # [ 0.870296] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk421vm-test-run-nixbot-gitlab> gitlab # [ 0.872616] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint422vm-test-run-nixbot-gitlab> gitlab # [ 0.877200] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]423vm-test-run-nixbot-gitlab> gitlab # [ 0.879908] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint424vm-test-run-nixbot-gitlab> gitlab # [ 0.883941] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI425vm-test-run-nixbot-gitlab> gitlab # [ 0.886311] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB426vm-test-run-nixbot-gitlab> gitlab # [ 0.888733] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint427vm-test-run-nixbot-gitlab> gitlab # [ 0.897399] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]428vm-test-run-nixbot-gitlab> gitlab # [ 0.899322] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]429vm-test-run-nixbot-gitlab> gitlab # [ 0.901348] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]430vm-test-run-nixbot-gitlab> gitlab # [ 0.902544] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]431vm-test-run-nixbot-gitlab> gitlab # [ 0.917310] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 13671 usecs432vm-test-run-nixbot-gitlab> gitlab # [ 0.920446] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint433vm-test-run-nixbot-gitlab> gitlab # [ 0.927585] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]434vm-test-run-nixbot-gitlab> gitlab # [ 0.929310] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]435vm-test-run-nixbot-gitlab> gitlab # [ 0.934436] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]436vm-test-run-nixbot-gitlab> gitlab # [ 0.939339] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]437vm-test-run-nixbot-gitlab> gitlab # [ 0.950675] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint438vm-test-run-nixbot-gitlab> gitlab # [ 0.962336] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]439vm-test-run-nixbot-gitlab> gitlab # [ 0.965329] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]440vm-test-run-nixbot-gitlab> gitlab # [ 0.969408] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]441vm-test-run-nixbot-gitlab> gitlab # [ 0.975361] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint442vm-test-run-nixbot-gitlab> gitlab # [ 0.981311] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]443vm-test-run-nixbot-gitlab> gitlab # [ 0.983310] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]444vm-test-run-nixbot-gitlab> gitlab # [ 0.985361] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]445vm-test-run-nixbot-gitlab> gitlab # [ 0.989823] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint446vm-test-run-nixbot-gitlab> gitlab # [ 0.995311] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]447vm-test-run-nixbot-gitlab> gitlab # [ 0.997310] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]448vm-test-run-nixbot-gitlab> gitlab # [ 0.998348] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]449vm-test-run-nixbot-gitlab> gitlab # [ 1.003360] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint450vm-test-run-nixbot-gitlab> gitlab # [ 1.008311] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]451vm-test-run-nixbot-gitlab> gitlab # [ 1.010311] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]452vm-test-run-nixbot-gitlab> gitlab # [ 1.012349] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]453vm-test-run-nixbot-gitlab> gitlab # [ 1.016647] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint454vm-test-run-nixbot-gitlab> gitlab # [ 1.022311] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]455vm-test-run-nixbot-gitlab> gitlab # [ 1.024311] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]456vm-test-run-nixbot-gitlab> gitlab # [ 1.026351] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]457vm-test-run-nixbot-gitlab> gitlab # [ 1.030336] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint458vm-test-run-nixbot-gitlab> gitlab # [ 1.035310] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]459vm-test-run-nixbot-gitlab> gitlab # [ 1.037311] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]460vm-test-run-nixbot-gitlab> gitlab # [ 1.038350] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]461vm-test-run-nixbot-gitlab> gitlab # [ 1.041340] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]462vm-test-run-nixbot-gitlab> gitlab # [ 1.045387] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint463vm-test-run-nixbot-gitlab> gitlab # [ 1.048895] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]464vm-test-run-nixbot-gitlab> gitlab # [ 1.051350] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]465vm-test-run-nixbot-gitlab> gitlab # [ 1.055322] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint466vm-test-run-nixbot-gitlab> gitlab # [ 1.061332] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]467vm-test-run-nixbot-gitlab> gitlab # [ 1.062312] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]468vm-test-run-nixbot-gitlab> gitlab # [ 1.064349] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]469vm-test-run-nixbot-gitlab> gitlab # [ 1.068290] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint470vm-test-run-nixbot-gitlab> gitlab # [ 1.074473] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]471vm-test-run-nixbot-gitlab> gitlab # [ 1.076311] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]472vm-test-run-nixbot-gitlab> gitlab # [ 1.077349] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]473vm-test-run-nixbot-gitlab> gitlab # [ 1.086867] ACPI: PCI: Interrupt link LNKA configured for IRQ 10474vm-test-run-nixbot-gitlab> gitlab # [ 1.089512] ACPI: PCI: Interrupt link LNKB configured for IRQ 10475vm-test-run-nixbot-gitlab> gitlab # [ 1.091497] ACPI: PCI: Interrupt link LNKC configured for IRQ 11476vm-test-run-nixbot-gitlab> gitlab # [ 1.093492] ACPI: PCI: Interrupt link LNKD configured for IRQ 11477vm-test-run-nixbot-gitlab> gitlab # [ 1.095408] ACPI: PCI: Interrupt link LNKS configured for IRQ 9478vm-test-run-nixbot-gitlab> gitlab # [ 1.098687] iommu: Default domain type: Translated479vm-test-run-nixbot-gitlab> gitlab # [ 1.100295] iommu: DMA domain TLB invalidation policy: lazy mode480vm-test-run-nixbot-gitlab> gitlab # [ 1.102660] ACPI: bus type USB registered481vm-test-run-nixbot-gitlab> gitlab # [ 1.104383] usbcore: registered new interface driver usbfs482vm-test-run-nixbot-gitlab> gitlab # [ 1.105321] usbcore: registered new interface driver hub483vm-test-run-nixbot-gitlab> gitlab # [ 1.107326] usbcore: registered new device driver usb484vm-test-run-nixbot-gitlab> gitlab # [ 1.109963] NetLabel: Initializing485vm-test-run-nixbot-gitlab> gitlab # [ 1.111296] NetLabel: domain hash size = 128486vm-test-run-nixbot-gitlab> gitlab # [ 1.112296] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO487vm-test-run-nixbot-gitlab> gitlab # [ 1.114391] NetLabel: unlabeled traffic allowed by default488vm-test-run-nixbot-gitlab> gitlab # [ 1.116311] PCI: Using ACPI for IRQ routing489vm-test-run-nixbot-gitlab> gitlab # [ 1.118773] pci 0000:00:02.0: vgaarb: setting as boot VGA device490vm-test-run-nixbot-gitlab> gitlab # [ 1.119290] pci 0000:00:02.0: vgaarb: bridge control possible491vm-test-run-nixbot-gitlab> gitlab # [ 1.119290] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none492vm-test-run-nixbot-gitlab> gitlab # [ 1.124316] vgaarb: loaded493vm-test-run-nixbot-gitlab> gitlab # [ 1.125766] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0494vm-test-run-nixbot-gitlab> gitlab # [ 1.127298] hpet0: 3 comparators, 64-bit 100.000000 MHz counter495vm-test-run-nixbot-gitlab> gitlab # [ 1.133579] clocksource: Switched to clocksource kvm-clock496vm-test-run-nixbot-gitlab> gitlab # [ 1.137773] VFS: Disk quotas dquot_6.6.0497vm-test-run-nixbot-gitlab> gitlab # [ 1.139042] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)498vm-test-run-nixbot-gitlab> gitlab # [ 1.141879] pnp: PnP ACPI init499vm-test-run-nixbot-gitlab> gitlab # [ 1.143836] pnp: PnP ACPI: found 6 devices500vm-test-run-nixbot-gitlab> gitlab # [ 1.155336] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns501vm-test-run-nixbot-gitlab> gitlab # [ 1.161976] clocksource: Switched to clocksource acpi_pm502vm-test-run-nixbot-gitlab> gitlab # [ 1.166650] NET: Registered PF_INET protocol family503vm-test-run-nixbot-gitlab> gitlab # [ 1.174181] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)504vm-test-run-nixbot-gitlab> gitlab # [ 1.210065] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)505vm-test-run-nixbot-gitlab> gitlab # [ 1.212696] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)506vm-test-run-nixbot-gitlab> gitlab # [ 1.215115] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)507vm-test-run-nixbot-gitlab> gitlab # [ 1.220159] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)508vm-test-run-nixbot-gitlab> gitlab # [ 1.222753] TCP: Hash tables configured (established 65536 bind 65536)509vm-test-run-nixbot-gitlab> gitlab # [ 1.227157] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)510vm-test-run-nixbot-gitlab> gitlab # [ 1.231788] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)511vm-test-run-nixbot-gitlab> gitlab # [ 1.235904] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)512vm-test-run-nixbot-gitlab> gitlab # [ 1.240726] NET: Registered PF_UNIX/PF_LOCAL protocol family513vm-test-run-nixbot-gitlab> gitlab # [ 1.244323] NET: Registered PF_XDP protocol family514vm-test-run-nixbot-gitlab> gitlab # [ 1.247593] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]515vm-test-run-nixbot-gitlab> gitlab # [ 1.251291] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]516vm-test-run-nixbot-gitlab> gitlab # [ 1.254997] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]517vm-test-run-nixbot-gitlab> gitlab # [ 1.259116] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]518vm-test-run-nixbot-gitlab> gitlab # [ 1.263190] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]519vm-test-run-nixbot-gitlab> gitlab # [ 1.266974] pci 0000:00:01.0: PIIX3: Enabling Passive Release520vm-test-run-nixbot-gitlab> gitlab # [ 1.268735] pci 0000:00:00.0: Limiting direct PCI/PCI transfers521vm-test-run-nixbot-gitlab> gitlab # [ 1.272133] ACPI: \_SB_.LNKD: Enabled at IRQ 11522vm-test-run-nixbot-gitlab> gitlab # [ 1.275685] PCI: CLS 0 bytes, default 64523vm-test-run-nixbot-gitlab> gitlab # [ 1.277088] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)524vm-test-run-nixbot-gitlab> gitlab # [ 1.277292] Trying to unpack rootfs image as initramfs...525vm-test-run-nixbot-gitlab> gitlab # [ 1.277403] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)526vm-test-run-nixbot-gitlab> gitlab # [ 1.283167] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns527vm-test-run-nixbot-gitlab> gitlab # [ 1.305688] Initialise system trusted keyrings528vm-test-run-nixbot-gitlab> gitlab # [ 1.307744] workingset: timestamp_bits=40 max_order=21 bucket_order=0529vm-test-run-nixbot-gitlab> gitlab # [ 1.324926] Key type asymmetric registered530vm-test-run-nixbot-gitlab> gitlab # [ 1.326252] Asymmetric key parser 'x509' registered531vm-test-run-nixbot-gitlab> gitlab # [ 1.327962] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)532vm-test-run-nixbot-gitlab> gitlab # [ 1.330782] io scheduler mq-deadline registered533vm-test-run-nixbot-gitlab> gitlab # [ 1.332215] io scheduler kyber registered534vm-test-run-nixbot-gitlab> gitlab # [ 1.337133] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled535vm-test-run-nixbot-gitlab> gitlab # [ 1.339541] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A536vm-test-run-nixbot-gitlab> gitlab # [ 1.343705] Linux agpgart interface v0.103537vm-test-run-nixbot-gitlab> gitlab # [ 1.345098] ACPI: bus type drm_connector registered538vm-test-run-nixbot-gitlab> gitlab # [ 1.349336] usbcore: registered new interface driver usbserial_generic539vm-test-run-nixbot-gitlab> gitlab # [ 1.351304] usbserial: USB Serial support registered for generic540vm-test-run-nixbot-gitlab> gitlab # [ 1.353261] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.541vm-test-run-nixbot-gitlab> gitlab # [ 1.353261] Please enable it if your BIOS has the CPPC option.542vm-test-run-nixbot-gitlab> gitlab # [ 1.364325] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled543vm-test-run-nixbot-gitlab> gitlab # [ 1.370846] drop_monitor: Initializing network drop monitor service544vm-test-run-nixbot-gitlab> gitlab # [ 1.376764] NET: Registered PF_INET6 protocol family545vm-test-run-nixbot-gitlab> gitlab # [ 1.384141] Segment Routing with IPv6546vm-test-run-nixbot-gitlab> gitlab # [ 1.387667] In-situ OAM (IOAM) with IPv6547vm-test-run-nixbot-gitlab> gitlab # [ 1.393290] IPI shorthand broadcast: enabled548vm-test-run-nixbot-gitlab> gitlab # [ 1.406523] sched_clock: Marking stable (1142079334, 264414732)->(1486923075, -80429009)549vm-test-run-nixbot-gitlab> gitlab # [ 1.413836] registered taskstats version 1550vm-test-run-nixbot-gitlab> gitlab # [ 1.418539] Loading compiled-in X.509 certificates551vm-test-run-nixbot-gitlab> gitlab # [ 1.442715] Demotion targets for Node 0: null552vm-test-run-nixbot-gitlab> gitlab # [ 1.444793] Key type .fscrypt registered553vm-test-run-nixbot-gitlab> gitlab # [ 1.446048] Key type fscrypt-provisioning registered554vm-test-run-nixbot-gitlab> gitlab # [ 1.447775] ima: No TPM chip found, activating TPM-bypass!555vm-test-run-nixbot-gitlab> gitlab # [ 1.449457] ima: Allocated hash algorithm: sha1556vm-test-run-nixbot-gitlab> gitlab # [ 1.450931] ima: No architecture policies found557vm-test-run-nixbot-gitlab> gitlab # [ 1.452746] PM: Magic number: 14:102:890558vm-test-run-nixbot-gitlab> gitlab # [ 1.455045] RAS: Correctable Errors collector initialized.559vm-test-run-nixbot-gitlab> gitlab # [ 1.460459] clk: Disabling unused clocks560vm-test-run-nixbot-gitlab> gitlab # [ 1.461728] PM: genpd: Disabling unused power domains561vm-test-run-nixbot-gitlab> gitlab # [ 1.549757] Freeing initrd memory: 27584K562vm-test-run-nixbot-gitlab> gitlab # [ 1.557023] Freeing unused decrypted memory: 2028K563vm-test-run-nixbot-gitlab> gitlab # [ 1.564037] Freeing unused kernel image (initmem) memory: 3644K564vm-test-run-nixbot-gitlab> gitlab # [ 1.567842] Write protecting the kernel read-only data: 32768k565vm-test-run-nixbot-gitlab> gitlab # [ 1.573942] Freeing unused kernel image (text/rodata gap) memory: 1332K566vm-test-run-nixbot-gitlab> gitlab # [ 1.579170] Freeing unused kernel image (rodata/data gap) memory: 816K567vm-test-run-nixbot-gitlab> gitlab # [ 1.640897] x86/mm: Checked W+X mappings: passed, no W+X pages found.568vm-test-run-nixbot-gitlab> gitlab # [ 1.642866] Run /init as init process569vm-test-run-nixbot-gitlab> gitlab # [ 1.659172] systemd[1]: Inserted module 'autofs4'570vm-test-run-nixbot-gitlab> gitlab # [ 1.688808] fuse: init (API version 7.45)571vm-test-run-nixbot-gitlab> gitlab # [ 1.698890] ACPI: \_SB_.LNKC: Enabled at IRQ 10572vm-test-run-nixbot-gitlab> gitlab # [ 1.710690] ACPI: \_SB_.LNKA: Enabled at IRQ 10573vm-test-run-nixbot-gitlab> gitlab # [ 1.715613] ACPI: \_SB_.LNKB: Enabled at IRQ 11574vm-test-run-nixbot-gitlab> gitlab # [ 1.778863] systemd[1]: Successfully made /usr/ read-only.575vm-test-run-nixbot-gitlab> gitlab # [ 2.116988] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)576vm-test-run-nixbot-gitlab> gitlab # [ 2.125823] systemd[1]: Detected virtualization kvm.577vm-test-run-nixbot-gitlab> gitlab # [ 2.127382] systemd[1]: Detected architecture x86-64.578vm-test-run-nixbot-gitlab> gitlab # [ 2.129038] systemd[1]: Running in initrd.579vm-test-run-nixbot-gitlab> gitlab # [ 2.131001] systemd[1]: Initializing machine ID from random generator.580vm-test-run-nixbot-gitlab> gitlab # [ 2.133232] systemd[1]: Hostname set to <gitlab>.581vm-test-run-nixbot-gitlab> gitlab # [ 2.189183] systemd[1]: Queued start job for default target Initrd Default Target.582vm-test-run-nixbot-gitlab> gitlab # [ 2.266932] systemd[1]: Created slice Slice /system/modprobe.583vm-test-run-nixbot-gitlab> gitlab # [ 2.269032] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.584vm-test-run-nixbot-gitlab> gitlab # [ 2.271523] systemd[1]: Expecting device /dev/disk/by-label/nix-store...585vm-test-run-nixbot-gitlab> gitlab # [ 2.273548] systemd[1]: Expecting device /dev/disk/by-label/nixos...586vm-test-run-nixbot-gitlab> gitlab # [ 2.275539] systemd[1]: Reached target Path Units.587vm-test-run-nixbot-gitlab> gitlab # [ 2.277128] systemd[1]: Reached target Slice Units.588vm-test-run-nixbot-gitlab> gitlab # [ 2.278741] systemd[1]: Reached target Swaps.589vm-test-run-nixbot-gitlab> gitlab # [ 2.280201] systemd[1]: Reached target Timer Units.590vm-test-run-nixbot-gitlab> gitlab # [ 2.281991] systemd[1]: Listening on D-Bus System Message Bus Socket.591vm-test-run-nixbot-gitlab> gitlab # [ 2.284132] systemd[1]: Listening on Journal Socket (/dev/log).592vm-test-run-nixbot-gitlab> gitlab # [ 2.286184] systemd[1]: Listening on Journal Sockets.593vm-test-run-nixbot-gitlab> gitlab # [ 2.288076] systemd[1]: Listening on udev Control Socket.594vm-test-run-nixbot-gitlab> gitlab # [ 2.289912] systemd[1]: Listening on udev Kernel Socket.595vm-test-run-nixbot-gitlab> gitlab # [ 2.291617] systemd[1]: Reached target Socket Units.596vm-test-run-nixbot-gitlab> gitlab # [ 2.295830] systemd[1]: Starting Create List of Static Device Nodes...597vm-test-run-nixbot-gitlab> gitlab # [ 2.300213] systemd[1]: Starting Load Kernel Module 9pnet_virtio...598vm-test-run-nixbot-gitlab> gitlab # [ 2.302361] systemd[1]: Starting Load Kernel Module configfs...599vm-test-run-nixbot-gitlab> gitlab # [ 2.310458] systemd[1]: Starting Journal Service...600vm-test-run-nixbot-gitlab> gitlab # [ 2.315400] systemd[1]: Starting Load Kernel Modules...601vm-test-run-nixbot-gitlab> gitlab # [ 2.317205] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki602vm-test-run-nixbot-gitlab> gitlab # [ 2.322247] systemd[1]: Starting Coldplug All udev Devices...603vm-test-run-nixbot-gitlab> gitlab # [ 2.341944] systemd-journald[152]: Collecting audit messages is disabled.604vm-test-run-nixbot-gitlab> gitlab # [ 2.342611] systemd[1]: Finished Create List of Static Device Nodes.605vm-test-run-nixbot-gitlab> gitlab # [ 2.349243] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.606vm-test-run-nixbot-gitlab> gitlab # [ 2.350333] systemd[1]: modprobe@configfs.service: Deactivated successfully.607vm-test-run-nixbot-gitlab> gitlab # [ 2.352514] netfs: FS-Cache loaded608vm-test-run-nixbot-gitlab> gitlab # [ 2.356595] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev609vm-test-run-nixbot-gitlab> gitlab # [ 2.359765] systemd[1]: Finished Load Kernel Module configfs.610vm-test-run-nixbot-gitlab> gitlab # [ 2.364443] 9pnet: Installing 9P2000 support611vm-test-run-nixbot-gitlab> gitlab # [ 2.365676] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config612vm-test-run-nixbot-gitlab> gitlab # [ 2.373014] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...613vm-test-run-nixbot-gitlab> gitlab # [ 2.376002] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.614vm-test-run-nixbot-gitlab> gitlab # [ 2.388936] systemd[1]: Finished Load Kernel Module 9pnet_virtio.615vm-test-run-nixbot-gitlab> gitlab # [ 2.395709] systemd[1]: Finished Load Kernel Modules.616vm-test-run-nixbot-gitlab> gitlab # [ 2.399670] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.617vm-test-run-nixbot-gitlab> gitlab # [ 2.405041] systemd[1]: Starting Apply Kernel Variables...618vm-test-run-nixbot-gitlab> gitlab # [ 2.414569] systemd[1]: Starting Create Static Device Nodes in /dev...619vm-test-run-nixbot-gitlab> gitlab # [ 2.426160] systemd[1]: Finished Apply Kernel Variables.620vm-test-run-nixbot-gitlab> gitlab # [ 2.161920] systemd-modules-load[154]: Using [ 2.428299] systemd[1]: Started Journal Service.621vm-test-run-nixbot-gitlab> gitlab # 4 probe threads622vm-test-run-nixbot-gitlab> gitlab # [ 2.166468] systemd-modules-load[154]: Inserted module 'virtio_balloon'623vm-test-run-nixbot-gitlab> gitlab # [ 2.168447] systemd-modules-load[154]: Inserted module 'virtio_gpu'624vm-test-run-nixbot-gitlab> gitlab # [ 2.170303] systemd-modules-load[154]: Inserted module 'dm_mod'625vm-test-run-nixbot-gitlab> gitlab # [ 2.177098] systemd[1]: Finished Create Static Device Nodes in /dev.626vm-test-run-nixbot-gitlab> gitlab # [ 2.178809] systemd[1]: Reached target Preparation for Local File Systems.627vm-test-run-nixbot-gitlab> gitlab # [ 2.180559] systemd[1]: Reached target Local File Systems.628vm-test-run-nixbot-gitlab> gitlab # [ 2.183128] systemd[1]: Starting Create System Files and Directories...629vm-test-run-nixbot-gitlab> gitlab # [ 2.190234] systemd[1]: Starting Rule-based Manager for Device Events and Files...630vm-test-run-nixbot-gitlab> gitlab # [ 2.200310] systemd[1]: Finished Create System Files and Directories.631vm-test-run-nixbot-gitlab> gitlab # [ 2.214933] systemd-udevd[194]: Using default interface naming scheme 'v260'.632vm-test-run-nixbot-gitlab> gitlab # [ 2.231441] systemd[1]: Started Rule-based Manager for Device Events and Files.633vm-test-run-nixbot-gitlab> gitlab # [ 2.241139] systemd[1]: Finished Coldplug All udev Devices.634vm-test-run-nixbot-gitlab> gitlab # [ 2.242725] systemd[1]: Reached target System Initialization.635vm-test-run-nixbot-gitlab> gitlab # [ 2.244254] systemd[1]: Reached target Basic System.636vm-test-run-nixbot-gitlab> gitlab # [ 2.670696] uhci_hcd 0000:00:01.2: UHCI Host Controller637vm-test-run-nixbot-gitlab> gitlab # [ 2.672580] virtio_blk virtio4: 4/0/0 default/read/poll queues638vm-test-run-nixbot-gitlab> gitlab # [ 2.672670] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1639vm-test-run-nixbot-gitlab> gitlab # [ 2.677283] uhci_hcd 0000:00:01.2: detected 2 ports640vm-test-run-nixbot-gitlab> gitlab # [ 2.682746] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140641vm-test-run-nixbot-gitlab> gitlab # [ 2.684775] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)642vm-test-run-nixbot-gitlab> gitlab # [ 2.685185] SCSI subsystem initialized643vm-test-run-nixbot-gitlab> gitlab # [ 2.686326] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18644vm-test-run-nixbot-gitlab> gitlab # [ 2.690561] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1645vm-test-run-nixbot-gitlab> gitlab # [ 2.692307] usb usb1: Product: UHCI Host Controller646vm-test-run-nixbot-gitlab> gitlab # [ 2.693573] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd647vm-test-run-nixbot-gitlab> gitlab # [ 2.694923] usb usb1: SerialNumber: 0000:00:01.2648vm-test-run-nixbot-gitlab> gitlab # [ 2.697826] hub 1-0:1.0: USB hub found649vm-test-run-nixbot-gitlab> gitlab # [ 2.699756] virtio_blk virtio5: 4/0/0 default/read/poll queues650vm-test-run-nixbot-gitlab> gitlab # [ 2.699910] hub 1-0:1.0: 2 ports detected651vm-test-run-nixbot-gitlab> gitlab # [ 2.711993] virtio_blk virtio5: [vdb] 11523456 512-byte logical blocks (5.90 GB/5.49 GiB)652vm-test-run-nixbot-gitlab> gitlab # [ 2.723728] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12653vm-test-run-nixbot-gitlab> gitlab # [ 2.731789] serio: i8042 KBD port at 0x60,0x64 irq 1654vm-test-run-nixbot-gitlab> gitlab # [ 2.733025] serio: i8042 AUX port at 0x60,0x64 irq 12655vm-test-run-nixbot-gitlab> gitlab # [ 2.749897] scsi host0: ata_piix656vm-test-run-nixbot-gitlab> gitlab # [ 2.755795] scsi host1: ata_piix657vm-test-run-nixbot-gitlab> gitlab # [ 2.756859] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0658vm-test-run-nixbot-gitlab> gitlab # [ 2.759638] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0659vm-test-run-nixbot-gitlab> gitlab # [ 2.503248] (udev-worker)[221]: Network interface NamePolicy= disabled on kernel command line.660vm-test-run-nixbot-gitlab> gitlab # [ 2.510129] (udev-worker)[206]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.661vm-test-run-nixbot-gitlab> gitlab # [ 2.514705] (udev-worker)[206]: Network interface NamePolicy= disabled on kernel command line.662vm-test-run-nixbot-gitlab> gitlab # [ 2.517270] systemd[1]: Starting Virtual Console Setup...663vm-test-run-nixbot-gitlab> gitlab # [ 2.532913] systemd[1]: Found device /dev/disk/by-label/nixos.664vm-test-run-nixbot-gitlab> gitlab # [ 2.536256] systemd[1]: Reached target Initrd Root Device.665vm-test-run-nixbot-gitlab> gitlab # [ 2.539075] systemd-vconsole-setup[233]: Configuration of first virtual console was skipped, ignoring remaining ones.666vm-test-run-nixbot-gitlab> gitlab # [ 2.542217] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...667vm-test-run-nixbot-gitlab> gitlab # [ 2.556311] systemd[1]: Finished Virtual Console Setup.668vm-test-run-nixbot-gitlab> gitlab # [ 2.561235] systemd[1]: Found device /dev/disk/by-label/nix-store.669vm-test-run-nixbot-gitlab> gitlab # [ 2.572306] systemd-fsck[241]: nixos: clean, 12/65536 files, 13019/262144 blocks670vm-test-run-nixbot-gitlab> gitlab # [ 2.579077] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.671vm-test-run-nixbot-gitlab> gitlab # [ 2.918733] ata2: found unknown device (class 0)672vm-test-run-nixbot-gitlab> gitlab # [ 2.920890] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100673vm-test-run-nixbot-gitlab> gitlab # [ 2.923973] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5674vm-test-run-nixbot-gitlab> gitlab # [ 2.934483] usb 1-1: new full-speed USB device number 2 using uhci_hcd675vm-test-run-nixbot-gitlab> gitlab # [ 2.951252] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0676vm-test-run-nixbot-gitlab> gitlab # [ 2.973467] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray677vm-test-run-nixbot-gitlab> gitlab # [ 3.000163] cdrom: Uniform CD-ROM driver Revision: 3.20678vm-test-run-nixbot-gitlab> gitlab # [ 3.100332] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00679vm-test-run-nixbot-gitlab> gitlab # [ 3.102475] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10680vm-test-run-nixbot-gitlab> gitlab # [ 3.104556] usb 1-1: Product: QEMU USB Tablet681vm-test-run-nixbot-gitlab> gitlab # [ 3.105837] usb 1-1: Manufacturer: QEMU682vm-test-run-nixbot-gitlab> gitlab # [ 3.106910] usb 1-1: SerialNumber: 28754-0000:00:01.2-1683vm-test-run-nixbot-gitlab> gitlab # [ 3.128151] hid: raw HID events driver (C) Jiri Kosina684vm-test-run-nixbot-gitlab> gitlab # [ 3.137064] usbcore: registered new interface driver usbhid685vm-test-run-nixbot-gitlab> gitlab # [ 3.138642] usbhid: USB HID core driver686vm-test-run-nixbot-gitlab> gitlab # [ 3.144589] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2687vm-test-run-nixbot-gitlab> gitlab # [ 3.147435] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0688vm-test-run-nixbot-gitlab> gitlab # [ 3.076758] systemd[1]: Mounting /sysroot...689vm-test-run-nixbot-gitlab> gitlab # [ 3.531212] EXT4-fs (vda): mounted filesystem bbd9d481-6050-4ef5-a9fc-6069af9bfffc r/w with ordered data mode. Quota mode: none.690vm-test-run-nixbot-gitlab> gitlab # [ 3.272317] systemd[1]: Mounted /sysroot.691vm-test-run-nixbot-gitlab> gitlab # [ 3.274301] systemd[1]: Reached target Initrd Root File System.692vm-test-run-nixbot-gitlab> gitlab # [ 3.276394] systemd[1]: Mounting /sysroot/nix/.ro-store...693vm-test-run-nixbot-gitlab> gitlab # [ 3.278990] systemd[1]: Mounting /sysroot/nix/.rw-store...694vm-test-run-nixbot-gitlab> gitlab # [ 3.290104] systemd[1]: Mounting /sysroot/run...695vm-test-run-nixbot-gitlab> gitlab # [ 3.293770] systemd[1]: Mounting /sysroot/tmp/shared...696vm-test-run-nixbot-gitlab> gitlab # [ 3.298137] systemd[1]: Mounting /sysroot/tmp/xchg...697vm-test-run-nixbot-gitlab> gitlab # [ 3.567665] erofs (device vdb): mounted with root inode @ nid 128.698vm-test-run-nixbot-gitlab> gitlab # [ 3.304419] systemd[1]: Starting Mountpoints Configured in the Real Root...699vm-test-run-nixbot-gitlab> gitlab # [ 3.577106] 9p: Installing v9fs 9p2000 file system support700vm-test-run-nixbot-gitlab> gitlab # [ 3.313202] systemd-sysroot-fstab-check[269]: /sysroot should be mounted in the initrd, will request daemon-reload.701vm-test-run-nixbot-gitlab> gitlab # [ 3.318920] systemd[1]: Mounted /sysroot/nix/.ro-store.702vm-test-run-nixbot-gitlab> gitlab # [ 3.320865] systemd[1]: Mounted /sysroot/nix/.rw-store.703vm-test-run-nixbot-gitlab> gitlab # [ 3.322154] systemd[1]: Mounted /sysroot/run.704vm-test-run-nixbot-gitlab> gitlab # [ 3.323259] systemd[1]: Mounted /sysroot/tmp/shared.705vm-test-run-nixbot-gitlab> gitlab # [ 3.324503] systemd[1]: Mounted /sysroot/tmp/xchg.706vm-test-run-nixbot-gitlab> gitlab # [ 3.329155] systemd[1]: Starting rw-sysroot-nix-store.service...707vm-test-run-nixbot-gitlab> gitlab # [ 3.330802] systemd[1]: Reload requested from client PID 269 ('systemd-sysroot') (unit initrd-parse-etc.service)...708vm-test-run-nixbot-gitlab> gitlab # [ 3.333352] systemd[1]: Reloading...709vm-test-run-nixbot-gitlab> gitlab # [ 3.524785] systemd[1]: Reloading finished in 194 ms.710vm-test-run-nixbot-gitlab> gitlab # [ 3.582717] systemd-sysroot-fstab-check[269]: Requesting initrd-fs.target/start/replace...711vm-test-run-nixbot-gitlab> gitlab # [ 3.673504] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.712vm-test-run-nixbot-gitlab> gitlab # [ 3.676636] systemd[1]: Finished rw-sysroot-nix-store.service.713vm-test-run-nixbot-gitlab> gitlab # [ 3.683133] systemd-sysroot-fstab-check[269]: Requesting swap.target/start/replace...714vm-test-run-nixbot-gitlab> gitlab # [ 3.690847] systemd[1]: Starting rw-sysroot-nix-store.service...715vm-test-run-nixbot-gitlab> gitlab # [ 3.693241] systemd[1]: initrd-parse-etc.service: Deactivated successfully.716vm-test-run-nixbot-gitlab> gitlab # [ 3.695937] systemd[1]: Finished Mountpoints Configured in the Real Root.717vm-test-run-nixbot-gitlab> gitlab # [ 3.698645] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.718vm-test-run-nixbot-gitlab> gitlab # [ 3.714954] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.719vm-test-run-nixbot-gitlab> gitlab # [ 3.717123] systemd[1]: Finished rw-sysroot-nix-store.service.720vm-test-run-nixbot-gitlab> gitlab # [ 4.080608] systemd[1]: Mounting /sysroot/nix/store...721vm-test-run-nixbot-gitlab> gitlab # [ 4.135930] systemd[1]: Mounted /sysroot/nix/store.722vm-test-run-nixbot-gitlab> gitlab # [ 4.139206] systemd[1]: Reached target Initrd File Systems.723vm-test-run-nixbot-gitlab> gitlab # [ 4.144243] systemd[1]: Starting Find NixOS closure...724vm-test-run-nixbot-gitlab> gitlab # [ 4.150137] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...725vm-test-run-nixbot-gitlab> gitlab # [ 4.184352] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.726vm-test-run-nixbot-gitlab> gitlab # [ 4.189155] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.727vm-test-run-nixbot-gitlab> gitlab # [ 4.212649] systemd[1]: Finished Find NixOS closure.728vm-test-run-nixbot-gitlab> gitlab # [ 4.216135] systemd[1]: Reached target Initrd Default Target.729vm-test-run-nixbot-gitlab> gitlab # [ 4.223720] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...730vm-test-run-nixbot-gitlab> gitlab # [ 4.263443] systemd[1]: Stopped target Initrd Default Target.731vm-test-run-nixbot-gitlab> gitlab # [ 4.267173] systemd[1]: Stopped target Basic System.732vm-test-run-nixbot-gitlab> gitlab # [ 4.270461] systemd[1]: Stopped target Initrd Root Device.733vm-test-run-nixbot-gitlab> gitlab # [ 4.274439] systemd[1]: Stopped target Path Units.734vm-test-run-nixbot-gitlab> gitlab # [ 4.277556] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.735vm-test-run-nixbot-gitlab> gitlab # [ 4.282265] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.736vm-test-run-nixbot-gitlab> gitlab # [ 4.286967] systemd[1]: Stopped target Slice Units.737vm-test-run-nixbot-gitlab> gitlab # [ 4.290255] systemd[1]: Stopped target Socket Units.738vm-test-run-nixbot-gitlab> gitlab # [ 4.293539] systemd[1]: Stopped target System Initialization.739vm-test-run-nixbot-gitlab> gitlab # [ 4.296929] systemd[1]: Stopped target Swaps.740vm-test-run-nixbot-gitlab> gitlab # [ 4.299725] systemd[1]: Stopped target Timer Units.741vm-test-run-nixbot-gitlab> gitlab # [ 4.302089] systemd[1]: dbus.socket: Deactivated successfully.742vm-test-run-nixbot-gitlab> gitlab # [ 4.304880] systemd[1]: Closed D-Bus System Message Bus Socket.743vm-test-run-nixbot-gitlab> gitlab # [ 4.307252] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.744vm-test-run-nixbot-gitlab> gitlab # [ 4.310342] systemd[1]: Stopped Find NixOS closure.745vm-test-run-nixbot-gitlab> gitlab # [ 4.312437] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio746vm-test-run-nixbot-gitlab> gitlab # [ 4.317144] systemd[1]: Starting rw-sysroot-nix-store.service...747vm-test-run-nixbot-gitlab> gitlab # [ 4.319577] systemd[1]: systemd-sysctl.service: Deactivated successfully.748vm-test-run-nixbot-gitlab> gitlab # [ 4.322410] systemd[1]: Stopped Apply Kernel Variables.749vm-test-run-nixbot-gitlab> gitlab # [ 4.324573] systemd[1]: systemd-modules-load.service: Deactivated successfully.750vm-test-run-nixbot-gitlab> gitlab # [ 4.326323] systemd[1]: Stopped Load Kernel Modules.751vm-test-run-nixbot-gitlab> gitlab # [ 4.327594] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.752vm-test-run-nixbot-gitlab> gitlab # [ 4.329533] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.753vm-test-run-nixbot-gitlab> gitlab # [ 4.332618] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.754vm-test-run-nixbot-gitlab> gitlab # [ 4.334393] systemd[1]: Stopped Create System Files and Directories.755vm-test-run-nixbot-gitlab> gitlab # [ 4.338334] systemd[1]: Stopped target Local File Systems.756vm-test-run-nixbot-gitlab> gitlab # [ 4.339759] systemd[1]: Stopped target Preparation for Local File Systems.757vm-test-run-nixbot-gitlab> gitlab # [ 4.341401] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.758vm-test-run-nixbot-gitlab> gitlab # [ 4.343182] systemd[1]: Stopped Coldplug All udev Devices.759vm-test-run-nixbot-gitlab> gitlab # [ 4.344807] systemd[1]: Stopping Rule-based Manager for Device Events and Files...760vm-test-run-nixbot-gitlab> gitlab # [ 4.346940] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.761vm-test-run-nixbot-gitlab> gitlab # [ 4.349121] systemd[1]: Stopped Virtual Console Setup.762vm-test-run-nixbot-gitlab> gitlab # [ 4.353744] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.763vm-test-run-nixbot-gitlab> gitlab # [ 4.357568] systemd[1]: Finished rw-sysroot-nix-store.service.764vm-test-run-nixbot-gitlab> gitlab # [ 4.363628] systemd[1]: initrd-cleanup.service: Deactivated successfully.765vm-test-run-nixbot-gitlab> gitlab # [ 4.366188] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.766vm-test-run-nixbot-gitlab> gitlab # [ 4.371227] systemd[1]: systemd-udevd.service: Deactivated successfully.767vm-test-run-nixbot-gitlab> gitlab # [ 4.375078] systemd[1]: Stopped Rule-based Manager for Device Events and Files.768vm-test-run-nixbot-gitlab> gitlab # [ 4.384671] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.769vm-test-run-nixbot-gitlab> gitlab # [ 4.386427] systemd[1]: Closed udev Control Socket.770vm-test-run-nixbot-gitlab> gitlab # [ 4.393103] systemd[1]: Starting Cleanup udev Database...771vm-test-run-nixbot-gitlab> gitlab # [ 4.394524] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.772vm-test-run-nixbot-gitlab> gitlab # [ 4.396400] systemd[1]: Stopped Create Static Device Nodes in /dev.773vm-test-run-nixbot-gitlab> gitlab # [ 4.400257] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.774vm-test-run-nixbot-gitlab> gitlab # [ 4.402180] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.775vm-test-run-nixbot-gitlab> gitlab # [ 4.406433] systemd[1]: kmod-static-nodes.service: Deactivated successfully.776vm-test-run-nixbot-gitlab> gitlab # [ 4.408126] systemd[1]: Stopped Create List of Static Device Nodes.777vm-test-run-nixbot-gitlab> gitlab # [ 4.422074] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.778vm-test-run-nixbot-gitlab> gitlab # [ 4.426067] systemd[1]: Finished Cleanup udev Database.779vm-test-run-nixbot-gitlab> gitlab # [ 4.431393] systemd[1]: Reached target Switch Root.780vm-test-run-nixbot-gitlab> gitlab # [ 4.435734] systemd[1]: Starting NixOS Activation...781vm-test-run-nixbot-gitlab> gitlab # [ 4.537856] initrd-nixos-activation-start[569]: booting system configuration /nix/store/qibpq3z8311i99pbi450wlcmbny123z4-nixos-system-gitlab-test782vm-test-run-nixbot-gitlab> gitlab # [ 4.572225] initrd-nixos-activation-start[569]: running activation script...783vm-test-run-nixbot-gitlab> gitlab # [ 4.763892] initrd-nixos-activation-start[592]: setting up /etc...784vm-test-run-nixbot-gitlab> gitlab # [ 4.882304] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.785vm-test-run-nixbot-gitlab> gitlab # [ 4.885454] systemd[1]: Finished NixOS Activation.786vm-test-run-nixbot-gitlab> gitlab # [ 4.892750] systemd[1]: Starting Switch Root...787vm-test-run-nixbot-gitlab> gitlab # [ 4.913657] systemd[1]: Switching root.788vm-test-run-nixbot-gitlab> gitlab # [ 5.462783] systemd-journald[152]: Received SIGTERM from PID 1 (systemd).789vm-test-run-nixbot-gitlab> gitlab # [ 5.583802] NET: Registered PF_VSOCK protocol family790vm-test-run-nixbot-gitlab> gitlab # [ 5.957921] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)791vm-test-run-nixbot-gitlab> gitlab # [ 5.975746] systemd[1]: Detected virtualization kvm.792vm-test-run-nixbot-gitlab> gitlab # [ 5.979267] systemd[1]: Detected architecture x86-64.793vm-test-run-nixbot-gitlab> gitlab # [ 5.982956] systemd[1]: Detected first boot.794vm-test-run-nixbot-gitlab> gitlab # [ 5.987338] systemd[1]: Initializing machine ID from random generator.795vm-test-run-nixbot-gitlab> gitlab # [ 6.135515] systemd[1]: bpf-restrict-fs: LSM BPF program attached796vm-test-run-nixbot-gitlab> gitlab # [ 6.196078] systemd[1]: Applying preset policy.797vm-test-run-nixbot-gitlab> gitlab # [ 6.364905] systemd[1]: Populated /etc with preset unit settings.798vm-test-run-nixbot-gitlab> gitlab # [ 6.511234] systemd[1]: initrd-switch-root.service: Deactivated successfully.799vm-test-run-nixbot-gitlab> gitlab # [ 6.518283] systemd[1]: Stopped initrd-switch-root.service.800vm-test-run-nixbot-gitlab> gitlab # [ 6.527740] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.801vm-test-run-nixbot-gitlab> gitlab # [ 6.534701] systemd[1]: Created slice Slice /system/getty.802vm-test-run-nixbot-gitlab> gitlab # [ 6.539128] systemd[1]: Created slice GitLab DevOps Platform Slice.803vm-test-run-nixbot-gitlab> gitlab # [ 6.543820] systemd[1]: Created slice User and Session Slice.804vm-test-run-nixbot-gitlab> gitlab # [ 6.547167] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.805vm-test-run-nixbot-gitlab> gitlab # [ 6.550841] systemd[1]: Started Forward Password Requests to Wall Directory Watch.806vm-test-run-nixbot-gitlab> gitlab # [ 6.552814] systemd[1]: Expecting device /dev/hvc0...807vm-test-run-nixbot-gitlab> gitlab # [ 6.554150] systemd[1]: Expecting device /dev/ttyS0...808vm-test-run-nixbot-gitlab> gitlab # [ 6.555553] systemd[1]: Reached target Local Encrypted Volumes.809vm-test-run-nixbot-gitlab> gitlab # [ 6.557109] systemd[1]: Stopped target initrd-fs.target.810vm-test-run-nixbot-gitlab> gitlab # [ 6.558538] systemd[1]: Stopped target initrd-root-fs.target.811vm-test-run-nixbot-gitlab> gitlab # [ 6.560024] systemd[1]: Stopped target initrd-switch-root.target.812vm-test-run-nixbot-gitlab> gitlab # [ 6.561688] systemd[1]: Reached target Virtual Machines and Containers.813vm-test-run-nixbot-gitlab> gitlab # [ 6.563436] systemd[1]: Reached target Path Units.814vm-test-run-nixbot-gitlab> gitlab # [ 6.564883] systemd[1]: Reached target Remote File Systems.815vm-test-run-nixbot-gitlab> gitlab # [ 6.566438] systemd[1]: Reached target Slice Units.816vm-test-run-nixbot-gitlab> gitlab # [ 6.567796] systemd[1]: Reached target Swaps.817vm-test-run-nixbot-gitlab> gitlab # [ 6.570371] systemd[1]: Listening on Process Core Dump Socket.818vm-test-run-nixbot-gitlab> gitlab # [ 6.572652] systemd[1]: Listening on Credential Encryption/Decryption.819vm-test-run-nixbot-gitlab> gitlab # [ 6.576199] systemd[1]: Starting Journal Log Access Socket...820vm-test-run-nixbot-gitlab> gitlab # [ 6.578452] systemd[1]: Listening on Journal Audit Socket.821vm-test-run-nixbot-gitlab> gitlab # [ 6.580228] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.822vm-test-run-nixbot-gitlab> gitlab # [ 6.582211] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki823vm-test-run-nixbot-gitlab> gitlab # [ 6.584712] systemd[1]: Listening on udev Control Socket.824vm-test-run-nixbot-gitlab> gitlab # [ 6.588191] systemd[1]: Mounting Huge Pages File System...825vm-test-run-nixbot-gitlab> gitlab # [ 6.591100] systemd[1]: Mounting POSIX Message Queue File System...826vm-test-run-nixbot-gitlab> gitlab # [ 6.594228] systemd[1]: Mounting Kernel Debug File System...827vm-test-run-nixbot-gitlab> gitlab # [ 6.598525] systemd[1]: Mounting Kernel Trace File System...828vm-test-run-nixbot-gitlab> gitlab # [ 6.606054] systemd[1]: Starting Create List of Static Device Nodes...829vm-test-run-nixbot-gitlab> gitlab # [ 6.607818] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio830vm-test-run-nixbot-gitlab> gitlab # [ 6.612930] systemd[1]: Starting Load Kernel Module configfs...831vm-test-run-nixbot-gitlab> gitlab # [ 6.614664] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm832vm-test-run-nixbot-gitlab> gitlab # [ 6.617363] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore833vm-test-run-nixbot-gitlab> gitlab # [ 6.620307] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse834vm-test-run-nixbot-gitlab> gitlab # [ 6.624269] systemd[1]: Mounting FUSE Control File System...835vm-test-run-nixbot-gitlab> gitlab # [ 6.629809] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67836vm-test-run-nixbot-gitlab> gitlab # [ 6.648936] systemd[1]: Starting Journal Service...837vm-test-run-nixbot-gitlab> gitlab # [ 6.652567] systemd[1]: Starting Load Kernel Modules...838vm-test-run-nixbot-gitlab> gitlab # [ 6.657542] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...839vm-test-run-nixbot-gitlab> gitlab # [ 6.661525] systemd[1]: Starting Remount Root and Kernel File Systems...840vm-test-run-nixbot-gitlab> gitlab # [ 6.663206] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki841vm-test-run-nixbot-gitlab> gitlab # [ 6.682546] systemd[1]: Starting Coldplug All udev Devices...842vm-test-run-nixbot-gitlab> gitlab # [ 6.683654] systemd-journald[789]: Collecting audit messages is enabled.843vm-test-run-nixbot-gitlab> gitlab # [ 6.689805] systemd[1]: Listening on Journal Log Access Socket.844vm-test-run-nixbot-gitlab> gitlab # [ 6.692219] systemd[1]: Mounted Huge Pages File System.845vm-test-run-nixbot-gitlab> gitlab # [ 6.693972] systemd[1]: Mounted POSIX Message Queue File System.846vm-test-run-nixbot-gitlab> gitlab # [ 6.695717] systemd[1]: Mounted Kernel Debug File System.847vm-test-run-nixbot-gitlab> gitlab # [ 6.697491] systemd[1]: Mounted Kernel Trace File System.848vm-test-run-nixbot-gitlab> gitlab # [ 6.701652] EXT4-fs (vda): re-mounted bbd9d481-6050-4ef5-a9fc-6069af9bfffc.849vm-test-run-nixbot-gitlab> gitlab # [ 6.702560] systemd[1]: Finished Create List of Static Device Nodes.850vm-test-run-nixbot-gitlab> gitlab # [ 6.705357] loop: module loaded851vm-test-run-nixbot-gitlab> gitlab # [ 6.706963] systemd[1]: modprobe@configfs.service: Deactivated successfully.852vm-test-run-nixbot-gitlab> gitlab # [ 6.443792] systemd[1]: Queued start job for default target Multi-User System[ 6.709622] systemd[1]: Finished Load Kernel Module configfs.853vm-test-run-nixbot-gitlab> gitlab # .854vm-test-run-nixbot-gitlab> gitlab # [ 6.711948] systemd[1]: Started Journal Service.855vm-test-run-nixbot-gitlab> gitlab # [ 6.447826] systemd[1]: Unnecessary job was removed for /dev/vdb.856vm-test-run-nixbot-gitlab> gitlab # [ 6.450863] systemd[1]: systemd-journald.service: Deactivated successfully.857vm-test-run-nixbot-gitlab> gitlab # [ 6.453089] systemd-modules-load[790]: Using 3 probe threads858vm-test-run-nixbot-gitlab> gitlab # [ 6.455685] systemd-modules-load[790]: Inserted module 'tls'859vm-test-run-nixbot-gitlab> gitlab # [ 6.457413] systemd-modules-load[790]: Inserted module 'loop'860vm-test-run-nixbot-gitlab> gitlab # [ 6.459415] systemd[1]: Mounted FUSE Control File System.861vm-test-run-nixbot-gitlab> gitlab # [ 6.461304] systemd[1]: Finished Load Kernel Modules.862vm-test-run-nixbot-gitlab> gitlab # [ 6.463077] systemd[1]: Finished Remount Root and Kernel File Systems.863vm-test-run-nixbot-gitlab> gitlab # [ 6.466567] systemd[1]: Mounting Kernel Configuration File System...864vm-test-run-nixbot-gitlab> gitlab # [ 6.468560] systemd[1]: Starting Firewall...865vm-test-run-nixbot-gitlab> gitlab # [ 6.470112] systemd[1]: Starting Flush Journal to Persistent Storage...866vm-test-run-nixbot-gitlab> gitlab # [ 6.471873] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore867vm-test-run-nixbot-gitlab> gitlab # [ 6.475101] systemd[1]: Starting Load/Save OS Random Seed...868vm-test-run-nixbot-gitlab> gitlab # [ 6.477168] systemd-oomd[792]: No swap; memory pressure usage will be degraded869vm-test-run-nixbot-gitlab> gitlab # [ 6.480097] systemd[1]: Starting Apply Kernel Variables...870vm-test-run-nixbot-gitlab> gitlab # [ 6.750293] systemd-journald[789]: Received client request to flush runtime journal.871vm-test-run-nixbot-gitlab> gitlab # [ 6.551136] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...872vm-test-run-nixbot-gitlab> gitlab # [ 6.553316] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki873vm-test-run-nixbot-gitlab> gitlab # [ 6.555648] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.874vm-test-run-nixbot-gitlab> gitlab # [ 6.557523] systemd[1]: Mounted Kernel Configuration File System.875vm-test-run-nixbot-gitlab> gitlab # [ 6.559335] systemd[1]: Finished Load/Save OS Random Seed.876vm-test-run-nixbot-gitlab> gitlab # [ 6.560904] systemd[1]: Reached target First Boot Complete.877vm-test-run-nixbot-gitlab> gitlab # [ 6.562289] systemd[1]: Finished Apply Kernel Variables.878vm-test-run-nixbot-gitlab> gitlab # [ 6.563616] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.879vm-test-run-nixbot-gitlab> gitlab # [ 6.566155] systemd[1]: Starting Create Static Device Nodes in /dev...880vm-test-run-nixbot-gitlab> gitlab # [ 6.568279] systemd[1]: Finished Create Static Device Nodes in /dev.881vm-test-run-nixbot-gitlab> gitlab # [ 6.569817] systemd[1]: Reached target Preparation for Local File Systems.882vm-test-run-nixbot-gitlab> gitlab # [ 6.571808] systemd[1]: Starting Rule-based Manager for Device Events and Files...883vm-test-run-nixbot-gitlab> gitlab # [ 6.573619] systemd[1]: Finished Flush Journal to Persistent Storage.884vm-test-run-nixbot-gitlab> gitlab # [ 6.594315] systemd-udevd[832]: Using default interface naming scheme 'v260'.885vm-test-run-nixbot-gitlab> gitlab # [ 6.632810] systemd[1]: Started Rule-based Manager for Device Events and Files.886vm-test-run-nixbot-gitlab> gitlab # [ 6.652216] systemd[1]: Finished Coldplug All udev Devices.887vm-test-run-nixbot-gitlab> gitlab # [ 6.763179] systemd[1]: Finished Firewall.888vm-test-run-nixbot-gitlab> gitlab # [ 6.765207] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse889vm-test-run-nixbot-gitlab> gitlab # [ 6.784132] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.890vm-test-run-nixbot-gitlab> gitlab # [ 6.794925] (udev-worker)[912]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.891vm-test-run-nixbot-gitlab> gitlab # [ 6.798989] (udev-worker)[912]: Network interface NamePolicy= disabled on kernel command line.892vm-test-run-nixbot-gitlab> gitlab # [ 6.803915] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.893vm-test-run-nixbot-gitlab> gitlab # [ 6.810751] (udev-worker)[914]: Network interface NamePolicy= disabled on kernel command line.894vm-test-run-nixbot-gitlab> gitlab # [ 6.818722] systemd[1]: Condition check resulted in Virtio network device being skipped.895vm-test-run-nixbot-gitlab> gitlab # [ 6.821765] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore896vm-test-run-nixbot-gitlab> gitlab # [ 6.824927] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67897vm-test-run-nixbot-gitlab> gitlab # [ 6.829921] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore898vm-test-run-nixbot-gitlab> gitlab # [ 6.833271] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki899vm-test-run-nixbot-gitlab> gitlab # [ 6.835664] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki900vm-test-run-nixbot-gitlab> gitlab # [ 7.164127] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3901vm-test-run-nixbot-gitlab> gitlab # [ 7.167488] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console902vm-test-run-nixbot-gitlab> gitlab # [ 7.175579] ACPI: button: Power Button [PWRF]903vm-test-run-nixbot-gitlab> gitlab # [ 7.183539] Console: switching to colour dummy device 80x25904vm-test-run-nixbot-gitlab> gitlab # [ 7.185743] [drm] Found bochs VGA, ID 0xb0c5.905vm-test-run-nixbot-gitlab> gitlab # [ 7.185745] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.906vm-test-run-nixbot-gitlab> gitlab # [ 7.192599] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0907vm-test-run-nixbot-gitlab> gitlab # [ 7.192623] rtc_cmos 00:05: RTC can wake from S4908vm-test-run-nixbot-gitlab> gitlab # [ 7.194086] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD909vm-test-run-nixbot-gitlab> gitlab # [ 7.194494] mousedev: PS/2 mouse device common for all mice910vm-test-run-nixbot-gitlab> gitlab # [ 7.198021] rtc_cmos 00:05: registered as rtc0911vm-test-run-nixbot-gitlab> gitlab # [ 7.200991] rtc_cmos 00:05: setting system clock to 2026-06-17T14:51:31 UTC (1781707891)912vm-test-run-nixbot-gitlab> gitlab # [ 7.203471] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic913vm-test-run-nixbot-gitlab> gitlab # [ 7.206039] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0914vm-test-run-nixbot-gitlab> gitlab # [ 7.206148] parport_pc 00:03: reported by Plug and Play ACPI915vm-test-run-nixbot-gitlab> gitlab # [ 7.209281] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs916vm-test-run-nixbot-gitlab> gitlab # [ 7.209985] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]917vm-test-run-nixbot-gitlab> gitlab # [ 7.214215] fbcon: bochs-drmdrmfb (fb0) is primary device918vm-test-run-nixbot-gitlab> gitlab # [ 7.214629] systemd-journald[789]: Time jumped backwards, rotating.919vm-test-run-nixbot-gitlab> gitlab # [ 7.221977] Console: switching to colour frame buffer device 160x50920vm-test-run-nixbot-gitlab> gitlab # [ 7.222087] Floppy drive(s): fd0 is 2.88M AMI BIOS921vm-test-run-nixbot-gitlab> gitlab # [ 7.228522] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device922vm-test-run-nixbot-gitlab> gitlab # [ 7.236069] FDC 0 is a S82078B923vm-test-run-nixbot-gitlab> gitlab # [ 7.268937] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5924vm-test-run-nixbot-gitlab> gitlab # [ 7.269527] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6925vm-test-run-nixbot-gitlab> gitlab # [ 7.280720] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4926vm-test-run-nixbot-gitlab> gitlab # [ 7.079333] systemd[1]: Starting Virtual Console Setup...927vm-test-run-nixbot-gitlab> gitlab # [ 7.090883] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.928vm-test-run-nixbot-gitlab> gitlab # [ 7.093368] systemd[1]: Stopped Virtual Console Setup.929vm-test-run-nixbot-gitlab> gitlab # [ 7.101150] systemd[1]: Starting Virtual Console Setup...930vm-test-run-nixbot-gitlab> gitlab # [ 7.398115] erofs (device vdb): failed to read inode meta block (nid: 109785581): -4931vm-test-run-nixbot-gitlab> gitlab # [ 7.401677] ppdev: user-space parallel port driver932vm-test-run-nixbot-gitlab> gitlab # [ 7.142665] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.933vm-test-run-nixbot-gitlab> gitlab # [ 7.148417] systemd[1]: Stopped Virtual Console Setup.934vm-test-run-nixbot-gitlab> gitlab # [ 7.157224] systemd[1]: Starting Virtual Console Setup...935vm-test-run-nixbot-gitlab> gitlab # [ 7.438593] kvm_amd: TSC scaling supported936vm-test-run-nixbot-gitlab> gitlab # [ 7.439374] kvm_amd: Nested Virtualization enabled937vm-test-run-nixbot-gitlab> gitlab # [ 7.440461] kvm_amd: Nested Paging enabled938vm-test-run-nixbot-gitlab> gitlab # [ 7.441224] kvm_amd: LBR virtualization supported939vm-test-run-nixbot-gitlab> gitlab # [ 7.442239] kvm_amd: Virtual VMLOAD VMSAVE supported940vm-test-run-nixbot-gitlab> gitlab # [ 7.443356] kvm_amd: Virtual GIF supported941vm-test-run-nixbot-gitlab> gitlab # [ 7.444249] kvm_amd: Virtual NMI enabled942vm-test-run-nixbot-gitlab> gitlab # [ 7.497281] EDAC MC: Ver: 3.0.0943vm-test-run-nixbot-gitlab> gitlab # [ 7.243269] systemd-vconsole-setup[963]: Configuration of first virtual console was skipped, ignoring remaining ones.944vm-test-run-nixbot-gitlab> gitlab # [ 7.246531] systemd[1]: Mounting /run/wrappers...945vm-test-run-nixbot-gitlab> gitlab # [ 7.247936] systemd[1]: Finished Virtual Console Setup.946vm-test-run-nixbot-gitlab> gitlab # [ 7.267126] systemd[1]: Mounted /run/wrappers.947vm-test-run-nixbot-gitlab> gitlab # [ 7.268528] systemd[1]: Reached target Local File Systems.948vm-test-run-nixbot-gitlab> gitlab # [ 7.269942] systemd[1]: Listening on Boot Loader Control Service Socket.949vm-test-run-nixbot-gitlab> gitlab # [ 7.271687] systemd[1]: Starting register-nix-paths.service...950vm-test-run-nixbot-gitlab> gitlab # [ 7.273386] systemd[1]: Starting Create SUID/SGID Wrappers...951vm-test-run-nixbot-gitlab> gitlab # [ 7.275109] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.952vm-test-run-nixbot-gitlab> gitlab # [ 7.277365] systemd[1]: Starting Save Transient machine-id to Disk...953vm-test-run-nixbot-gitlab> gitlab # [ 7.283229] systemd[1]: Starting Create System Files and Directories...954vm-test-run-nixbot-gitlab> gitlab # [ 7.308169] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.955vm-test-run-nixbot-gitlab> gitlab # [ 7.310298] systemd[1]: Finished Save Transient machine-id to Disk.956vm-test-run-nixbot-gitlab> gitlab # [ 7.320462] systemd[1]: Finished Create System Files and Directories.957vm-test-run-nixbot-gitlab> gitlab # [ 7.323821] systemd[1]: Starting Rebuild Journal Catalog...958vm-test-run-nixbot-gitlab> gitlab # [ 7.326258] systemd[1]: Starting Record System Boot/Shutdown in UTMP...959vm-test-run-nixbot-gitlab> gitlab # [ 7.345875] systemd[1]: Finished Record System Boot/Shutdown in UTMP.960vm-test-run-nixbot-gitlab> gitlab # [ 7.362441] systemd[1]: Finished Rebuild Journal Catalog.961vm-test-run-nixbot-gitlab> gitlab # [ 7.364735] systemd[1]: Starting Update is Completed...962vm-test-run-nixbot-gitlab> gitlab # [ 7.381101] systemd[1]: Finished Update is Completed.963vm-test-run-nixbot-gitlab> gitlab # [ 7.418540] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.964vm-test-run-nixbot-gitlab> gitlab # [ 7.420464] systemd[1]: Finished Create SUID/SGID Wrappers.965vm-test-run-nixbot-gitlab> gitlab # [ 7.422201] systemd[1]: suid-sgid-wrappers.service: Consumed 122ms CPU time over 146ms wall clock time, 4.3M memory peak, 1.1M read from disk.966vm-test-run-nixbot-gitlab> gitlab # [ 7.685985] systemd[1]: Finished register-nix-paths.service.967vm-test-run-nixbot-gitlab> gitlab # [ 7.689993] systemd[1]: Reached target System Initialization.968vm-test-run-nixbot-gitlab> gitlab # [ 7.693989] systemd[1]: Started Discard unused filesystem blocks once a week.969vm-test-run-nixbot-gitlab> gitlab # [ 7.698685] systemd[1]: Started logrotate.timer.970vm-test-run-nixbot-gitlab> gitlab # [ 7.702201] systemd[1]: Started Daily Cleanup of Temporary Directories.971vm-test-run-nixbot-gitlab> gitlab # [ 7.706565] systemd[1]: Reached target Timer Units.972vm-test-run-nixbot-gitlab> gitlab # [ 7.710187] systemd[1]: Listening on D-Bus System Message Bus Socket.973vm-test-run-nixbot-gitlab> gitlab # [ 7.714439] systemd[1]: Listening on Nix Daemon Socket.974vm-test-run-nixbot-gitlab> gitlab # [ 7.718318] systemd[1]: Starting nixbot CI service socket...975vm-test-run-nixbot-gitlab> gitlab # [ 7.722457] systemd[1]: Listening on Hostname Service Socket.976vm-test-run-nixbot-gitlab> gitlab # [ 7.727193] systemd[1]: Starting D-Bus System Message Bus...977vm-test-run-nixbot-gitlab> gitlab # [ 7.731122] systemd[1]: Listening on nixbot CI service socket.978vm-test-run-nixbot-gitlab> gitlab # [ 7.734307] systemd[1]: Reached target Socket Units.979vm-test-run-nixbot-gitlab> gitlab # [ 7.770494] dbus-broker-launch[1054]: Looking up NSS user entry for 'systemd-timesync'...980vm-test-run-nixbot-gitlab> gitlab # [ 7.775249] dbus-broker-launch[1054]: NSS returned no entry for 'systemd-timesync'981vm-test-run-nixbot-gitlab> gitlab # [ 7.778942] dbus-broker-launch[1054]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"982vm-test-run-nixbot-gitlab> gitlab # [ 7.787485] systemd[1]: Started D-Bus System Message Bus.983vm-test-run-nixbot-gitlab> gitlab # [ 7.803901] dbus-broker-launch[1054]: Ready984vm-test-run-nixbot-gitlab> gitlab # [ 7.810189] systemd[1]: Reached target Basic System.985vm-test-run-nixbot-gitlab> gitlab # [ 7.813936] systemd[1]: Started backdoor.service.986vm-test-run-nixbot-gitlab> gitlab # [ 7.818501] systemd[1]: Starting gitlab-config.service...987vm-test-run-nixbot-gitlab> gitlab # [ 7.829154] systemd[1]: Starting Import lastlog data into lastlog2 database...988vm-test-run-nixbot-gitlab> gitlab # [ 7.838484] systemd[1]: Starting Logrotate configuration check...989vm-test-run-nixbot-gitlab> gitlab # [ 7.843106] systemd[1]: Starting Name Service Cache Daemon (nsncd)...990vm-test-run-nixbot-gitlab> gitlab # [ 7.846359] systemd[1]: Starting Post-Boot Actions...991vm-test-run-nixbot-gitlab> gitlab # [ 7.855061] systemd[1]: Started Reset console on configuration changes.992vm-test-run-nixbot-gitlab> gitlab # [ 7.858275] systemd[1]: Starting resolvconf update...993vm-test-run-nixbot-gitlab> gitlab # [ 7.860808] logrotate[1059]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.994vm-test-run-nixbot-gitlab> gitlab # [ 7.867150] logrotate[1059]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf995vm-test-run-nixbot-gitlab> gitlab # connecting to host...996vm-test-run-nixbot-gitlab> gitlab # [ 7.869310] logrotate[1059]: note: 'monthly' overrides previously specified 'weekly'997vm-test-run-nixbot-gitlab> gitlab # [ 7.873722] logrotate[1059]: Reading state from file: /var/lib/logrotate.status998vm-test-run-nixbot-gitlab> gitlab # [ 7.875972] logrotate[1059]: state file /var/lib/logrotate.status does not exist999vm-test-run-nixbot-gitlab> gitlab # [ 7.877518] logrotate[1059]: Allocating hash table for state file, size 64 entries1000vm-test-run-nixbot-gitlab> gitlab # [ 7.878560] logrotate[1059]: Handling 4 logs1001vm-test-run-nixbot-gitlab> gitlab # [ 7.879966] logrotate[1059]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1002vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'1003vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell1004vm-test-run-nixbot-gitlab> gitlab # [ 7.881340] logrotate[1059]: considering log /var/log/btmp1005vm-test-run-nixbot-gitlab> gitlab: (connecting took 8.72 seconds)1006vm-test-run-nixbot-gitlab> gitlab # [ 7.882249] logrotate[1059]: Creating new state1007vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 8.72 seconds)1008vm-test-run-nixbot-gitlab> gitlab # [ 7.883175] logrotate[1059]: Now: 2026-06-17 14:511009vm-test-run-nixbot-gitlab> gitlab # [ 7.884109] logrotate[1059]: Last rotated at 2026-06-17 14:001010vm-test-run-nixbot-gitlab> gitlab # [ 7.884996] logrotate[1059]: log does not need rotating (log has already been rotated)1011vm-test-run-nixbot-gitlab> gitlab # [ 7.886068] logrotate[1059]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1012vm-test-run-nixbot-gitlab> gitlab # [ 7.886960] logrotate[1059]: considering log /var/log/wtmp1013vm-test-run-nixbot-gitlab> gitlab # [ 7.887971] logrotate[1059]: Creating new state1014vm-test-run-nixbot-gitlab> gitlab # [ 7.889163] logrotate[1059]: Now: 2026-06-17 14:511015vm-test-run-nixbot-gitlab> gitlab # [ 7.889978] logrotate[1059]: Last rotated at 2026-06-17 14:001016vm-test-run-nixbot-gitlab> gitlab # [ 7.891173] logrotate[1059]: log does not need rotating (log has already been rotated)1017vm-test-run-nixbot-gitlab> gitlab # [ 7.891967] logrotate[1059]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed1018vm-test-run-nixbot-gitlab> gitlab # [ 7.894533] logrotate[1059]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1059)1019vm-test-run-nixbot-gitlab> gitlab # [ 7.895828] logrotate[1059]: considering log /var/gitlab/state/log/*.log1020vm-test-run-nixbot-gitlab> gitlab # [ 7.896863] logrotate[1059]: log /var/gitlab/state/log/*.log does not exist -- skipping1021vm-test-run-nixbot-gitlab> gitlab # [ 7.898036] logrotate[1059]: Creating new state1022vm-test-run-nixbot-gitlab> gitlab # [ 7.898643] logrotate[1059]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1059)1023vm-test-run-nixbot-gitlab> gitlab # [ 7.899861] logrotate[1059]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed1024vm-test-run-nixbot-gitlab> gitlab # [ 7.901064] logrotate[1059]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1059)1025vm-test-run-nixbot-gitlab> gitlab # [ 7.901929] logrotate[1059]: considering log /var/log/nginx/*.log1026vm-test-run-nixbot-gitlab> gitlab # [ 7.903140] logrotate[1059]: log /var/log/nginx/*.log does not exist -- skipping1027vm-test-run-nixbot-gitlab> gitlab # [ 7.904123] logrotate[1059]: Creating new state1028vm-test-run-nixbot-gitlab> gitlab # [ 7.904986] logrotate[1059]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1059)1029vm-test-run-nixbot-gitlab> gitlab # [ 7.911805] systemd[1]: Finished Logrotate configuration check.1030vm-test-run-nixbot-gitlab> gitlab # [ 7.914322] systemd[1]: Finished Post-Boot Actions.1031vm-test-run-nixbot-gitlab> gitlab # [ 7.925174] systemd[1]: Finished Import lastlog data into lastlog2 database.1032vm-test-run-nixbot-gitlab> gitlab # [ 7.942764] nsncd[1060]: Jun 17 14:51:32.504 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1033vm-test-run-nixbot-gitlab> gitlab # [ 7.946655] systemd[1]: Started Name Service Cache Daemon (nsncd).1034vm-test-run-nixbot-gitlab> gitlab # [ 7.948380] systemd[1]: Reached target Host and Network Name Lookups.1035vm-test-run-nixbot-gitlab> gitlab # [ 7.950141] systemd[1]: Reached target User and Group Name Lookups.1036vm-test-run-nixbot-gitlab> gitlab # [ 7.951892] systemd[1]: Starting User Login Management...1037vm-test-run-nixbot-gitlab> gitlab # [ 7.976818] systemd[1]: Stopped target Host and Network Name Lookups.1038vm-test-run-nixbot-gitlab> gitlab # [ 7.978083] systemd[1]: Stopping Host and Network Name Lookups...1039vm-test-run-nixbot-gitlab> gitlab # [ 7.978921] systemd[1]: Stopped target User and Group Name Lookups.1040vm-test-run-nixbot-gitlab> gitlab # [ 7.982543] systemd[1]: Stopping User and Group Name Lookups...1041vm-test-run-nixbot-gitlab> gitlab # [ 7.984365] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...1042vm-test-run-nixbot-gitlab> gitlab # [ 7.986184] systemd[1]: nscd.service: Deactivated successfully.1043vm-test-run-nixbot-gitlab> gitlab # [ 7.987854] systemd[1]: Stopped Name Service Cache Daemon (nsncd).1044vm-test-run-nixbot-gitlab> gitlab # [ 7.989648] systemd[1]: nscd.service: Consumed 15ms CPU time over 139ms wall clock time, 8M memory peak, 5.6M read from disk.1045vm-test-run-nixbot-gitlab> gitlab # [ 7.992837] systemd[1]: Starting Name Service Cache Daemon (nsncd)...1046vm-test-run-nixbot-gitlab> gitlab # [ 8.008871] systemd-logind[1116]: New seat seat0.1047vm-test-run-nixbot-gitlab> gitlab # [ 8.010676] systemd[1]: Started Name Service Cache Daemon (nsncd).1048vm-test-run-nixbot-gitlab> gitlab # [ 8.017058] systemd[1]: Reached target Host and Network Name Lookups.1049vm-test-run-nixbot-gitlab> gitlab # [ 8.018959] nsncd[1144]: Jun 17 14:51:32.577 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1050vm-test-run-nixbot-gitlab> gitlab # [ 8.020168] systemd[1]: Reached target User and Group Name Lookups.1051vm-test-run-nixbot-gitlab> gitlab # [ 8.022399] systemd-logind[1116]: Watching system buttons on /dev/input/event2 (Power Button)1052vm-test-run-nixbot-gitlab> gitlab # [ 8.024388] systemd-logind[1116]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)1053vm-test-run-nixbot-gitlab> gitlab # [ 8.025931] systemd-logind[1116]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)1054vm-test-run-nixbot-gitlab> gitlab # [ 8.027137] systemd[1]: Started User Login Management.1055vm-test-run-nixbot-gitlab> gitlab # [ 8.033979] systemd[1]: Starting linger-users.service...1056vm-test-run-nixbot-gitlab> gitlab # [ 8.043243] systemd[1]: Finished resolvconf update.1057vm-test-run-nixbot-gitlab> gitlab # [ 8.044923] systemd[1]: Reached target Preparation for Network.1058vm-test-run-nixbot-gitlab> gitlab # [ 8.048748] systemd[1]: Starting DHCP Client...1059vm-test-run-nixbot-gitlab> gitlab # [ 8.050281] systemd[1]: Starting Address configuration of eth1...1060vm-test-run-nixbot-gitlab> gitlab # [ 8.053215] systemd[1]: Starting Extra networking commands....1061vm-test-run-nixbot-gitlab> gitlab # [ 8.055225] systemd[1]: linger-users.service: Deactivated successfully.1062vm-test-run-nixbot-gitlab> gitlab # [ 8.056453] systemd[1]: Finished linger-users.service.1063vm-test-run-nixbot-gitlab> gitlab # [ 8.085137] network-addresses-eth1-start[1171]: adding address 192.168.1.1/24... done1064vm-test-run-nixbot-gitlab> gitlab # [ 8.089433] network-addresses-eth1-start[1171]: adding address 2001:db8:1::1/64... done1065vm-test-run-nixbot-gitlab> gitlab # [ 8.097555] systemd[1]: Finished Address configuration of eth1.1066vm-test-run-nixbot-gitlab> gitlab # [ 8.126163] systemd[1]: Finished Extra networking commands..1067vm-test-run-nixbot-gitlab> gitlab # [ 8.127748] systemd[1]: Reached target Network.1068vm-test-run-nixbot-gitlab> gitlab # [ 8.129622] systemd[1]: Starting gitlab-workhorse.service...1069vm-test-run-nixbot-gitlab> gitlab # [ 8.136066] systemd[1]: Starting Nginx Web Server...1070vm-test-run-nixbot-gitlab> gitlab # [ 8.138666] systemd[1]: Starting PostgreSQL Server...1071vm-test-run-nixbot-gitlab> gitlab # [ 8.140733] systemd[1]: Starting Redis Server - redis-gitlab...1072vm-test-run-nixbot-gitlab> gitlab # [ 8.144600] systemd[1]: Starting Permit User Sessions...1073vm-test-run-nixbot-gitlab> gitlab # [ 8.148640] dhcpcd[1197]: dhcpcd-10.3.2 starting1074vm-test-run-nixbot-gitlab> gitlab # [ 8.154323] dhcpcd[1235]: dev: loaded udev1075vm-test-run-nixbot-gitlab> gitlab # [ 8.424704] 8021q: 802.1Q VLAN Support v1.81076vm-test-run-nixbot-gitlab> gitlab # [ 8.425791] 8021q: adding VLAN 0 to HW filter on device eth11077vm-test-run-nixbot-gitlab> gitlab # [ 8.166328] systemd[1]: Finished Permit User Sessions.1078vm-test-run-nixbot-gitlab> gitlab # [ 8.171077] systemd[1]: Started Getty on tty1.1079vm-test-run-nixbot-gitlab> gitlab # [ 8.174748] systemd[1]: Reached target Login Prompts.1080vm-test-run-nixbot-gitlab> gitlab # [ 8.191911] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1081vm-test-run-nixbot-gitlab> gitlab # [ 8.537708] cfg80211: Loading compiled-in X.509 certificates for regulatory database1082vm-test-run-nixbot-gitlab> gitlab # [ 8.276419] nginx-pre-start[1255]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok1083vm-test-run-nixbot-gitlab> gitlab # [ 8.281052] nginx-pre-start[1255]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5ym[ 8.547051] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'1084vm-test-run-nixbot-gitlab> gitlab # l2qxbm30d7w-nginx.conf test is s[ 8.548765] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'1085vm-test-run-nixbot-gitlab> gitlab # uccessful1086vm-test-run-nixbot-gitlab> gitlab # [ 8.552064] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -21087vm-test-run-nixbot-gitlab> gitlab # [ 8.554100] cfg80211: failed to load regulatory.db1088vm-test-run-nixbot-gitlab> gitlab # [ 8.296376] systemd[1]: Started Nginx Web Server.1089vm-test-run-nixbot-gitlab> gitlab # [ 8.310154] postgresql-pre-start[1258]: The files belonging to this database system will be owned by user "postgres".1090vm-test-run-nixbot-gitlab> gitlab # [ 8.311280] postgresql-pre-start[1258]: This user must also own the server process.1091vm-test-run-nixbot-gitlab> gitlab # [ 8.317684] postgresql-pre-start[1258]: The database cluster will be initialized with locale "en_US.UTF-8".1092vm-test-run-nixbot-gitlab> gitlab # [ 8.318702] postgresql-pre-start[1258]: The default database encoding has accordingly been set to "UTF8".1093vm-test-run-nixbot-gitlab> gitlab # [ 8.319896] postgresql-pre-start[1258]: The default text search configuration will be set to "english".1094vm-test-run-nixbot-gitlab> gitlab # [ 8.321251] postgresql-pre-start[1258]: Data page checksums are disabled.1095vm-test-run-nixbot-gitlab> gitlab # [ 8.321901] postgresql-pre-start[1258]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1096vm-test-run-nixbot-gitlab> gitlab # [ 8.323320] postgresql-pre-start[1258]: creating subdirectories ... ok1097vm-test-run-nixbot-gitlab> gitlab # [ 8.324928] postgresql-pre-start[1258]: selecting dynamic shared memory implementation ... posix1098vm-test-run-nixbot-gitlab> gitlab # [ 8.326361] redis-server[1254]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1099vm-test-run-nixbot-gitlab> gitlab # [ 8.327397] redis-server[1254]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1100vm-test-run-nixbot-gitlab> gitlab # [ 8.328588] redis-server[1254]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1254, just started1101vm-test-run-nixbot-gitlab> gitlab # [ 8.329904] redis-server[1254]: Configuration loaded1102vm-test-run-nixbot-gitlab> gitlab # [ 8.331159] systemd[1]: Started Redis Server - redis-gitlab.1103vm-test-run-nixbot-gitlab> gitlab # [ 8.331985] redis[1254]: monotonic clock: POSIX clock_gettime1104vm-test-run-nixbot-gitlab> gitlab # [ 8.333164] redis[1254]: Running mode=standalone, port=0.1105vm-test-run-nixbot-gitlab> gitlab # [ 8.333854] redis[1254]: Server initialized1106vm-test-run-nixbot-gitlab> gitlab # [ 8.334922] redis[1254]: Ready to accept connections unix1107vm-test-run-nixbot-gitlab> gitlab # [ 8.336108] redis[1254]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1108vm-test-run-nixbot-gitlab> gitlab # [ 8.621938] 8021q: adding VLAN 0 to HW filter on device eth01109vm-test-run-nixbot-gitlab> gitlab # [ 8.360275] dhcpcd[1235]: eth0: waiting for carrier1110vm-test-run-nixbot-gitlab> gitlab # [ 8.364840] postgresql-pre-start[1258]: selecting default "max_connections" ... 1001111vm-test-run-nixbot-gitlab> gitlab # [ 8.404157] postgresql-pre-start[1258]: selecting default "shared_buffers" ... 128MB1112vm-test-run-nixbot-gitlab> gitlab # [ 8.614256] systemd[1]: Started gitlab-workhorse.service.1113vm-test-run-nixbot-gitlab> gitlab # [ 8.788432] postgresql-pre-start[1258]: selecting default time zone ... UTC1114vm-test-run-nixbot-gitlab> gitlab # [ 8.790707] postgresql-pre-start[1258]: creating configuration files ... ok1115vm-test-run-nixbot-gitlab> gitlab # [ 8.855359] gitlab-workhorse[1289]: time="2026-06-17T14:51:33Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41116vm-test-run-nixbot-gitlab> gitlab # [ 8.857173] gitlab-workhorse[1289]: time="2026-06-17T14:51:33Z" level=info msg="Using redis/go-redis"1117vm-test-run-nixbot-gitlab> gitlab # [ 8.858598] gitlab-workhorse[1289]: time="2026-06-17T14:51:33Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1118vm-test-run-nixbot-gitlab> gitlab # [ 8.980802] postgresql-pre-start[1258]: running bootstrap script ... ok1119vm-test-run-nixbot-gitlab> gitlab # [ 9.127791] dhcpcd[1235]: eth0: carrier acquired1120vm-test-run-nixbot-gitlab> gitlab # [ 9.135611] dhcpcd[1235]: DUID 00:01:00:01:31:c5:70:f5:52:54:00:12:34:561121vm-test-run-nixbot-gitlab> gitlab # [ 9.137506] dhcpcd[1235]: eth0: IAID 00:12:34:561122vm-test-run-nixbot-gitlab> gitlab # [ 9.139656] dhcpcd[1235]: eth0: adding address fe80::5054:ff:fe12:34561123vm-test-run-nixbot-gitlab> gitlab # [ 9.450900] postgresql-pre-start[1258]: performing post-bootstrap initialization ... ok1124vm-test-run-nixbot-gitlab> gitlab # [ 9.769632] dhcpcd[1235]: eth0: soliciting a DHCP lease1125vm-test-run-nixbot-gitlab> gitlab # [ 10.050732] NET: Registered PF_PACKET protocol family1126vm-test-run-nixbot-gitlab> gitlab # [ 9.794091] dhcpcd[1235]: eth0: offered 10.0.2.15 from 10.0.2.21127vm-test-run-nixbot-gitlab> gitlab # [ 9.801394] dhcpcd[1235]: eth0: probing address 10.0.2.15/241128vm-test-run-nixbot-gitlab> gitlab # [ 9.830313] postgresql-pre-start[1258]: syncing data to disk ... ok1129vm-test-run-nixbot-gitlab> gitlab # [ 9.831410] postgresql-pre-start[1258]: initdb: warning: enabling "trust" authentication for local connections1130vm-test-run-nixbot-gitlab> gitlab # [ 9.832300] postgresql-pre-start[1258]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1131vm-test-run-nixbot-gitlab> gitlab # [ 9.833202] postgresql-pre-start[1258]: Success. You can now start the database server using:1132vm-test-run-nixbot-gitlab> gitlab # [ 9.834081] postgresql-pre-start[1258]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1133vm-test-run-nixbot-gitlab> gitlab # [ 9.918219] postgres[1317]: [1317] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1134vm-test-run-nixbot-gitlab> gitlab # [ 9.926311] postgres[1317]: [1317] LOG: listening on IPv6 address "::1", port 54321135vm-test-run-nixbot-gitlab> gitlab # [ 9.931750] postgres[1317]: [1317] LOG: listening on IPv4 address "127.0.0.1", port 54321136vm-test-run-nixbot-gitlab> gitlab # [ 9.940779] postgres[1317]: [1317] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1137vm-test-run-nixbot-gitlab> gitlab # [ 9.946325] postgres[1323]: [1323] LOG: database system was shut down at 2026-06-17 14:51:34 GMT1138vm-test-run-nixbot-gitlab> gitlab # [ 9.951519] postgres[1317]: [1317] LOG: database system is ready to accept connections1139vm-test-run-nixbot-gitlab> gitlab # [ 9.956373] systemd[1]: Started PostgreSQL Server.1140vm-test-run-nixbot-gitlab> gitlab # [ 9.960792] systemd[1]: Starting PostgreSQL Setup Scripts...1141vm-test-run-nixbot-gitlab> gitlab # [ 10.117218] postgresql-setup-start[1334]: CREATE DATABASE1142vm-test-run-nixbot-gitlab> gitlab # [ 10.147590] postgresql-setup-start[1352]: CREATE ROLE1143vm-test-run-nixbot-gitlab> gitlab # [ 10.157515] postgresql-setup-start[1354]: ALTER DATABASE1144vm-test-run-nixbot-gitlab> gitlab # [ 10.186837] postgresql-setup-start[1359]: CREATE ROLE1145vm-test-run-nixbot-gitlab> gitlab # [ 10.190804] systemd[1]: Finished PostgreSQL Setup Scripts.1146vm-test-run-nixbot-gitlab> gitlab # [ 10.192104] systemd[1]: Reached target PostgreSQL.1147vm-test-run-nixbot-gitlab> gitlab # [ 10.193611] systemd[1]: Starting gitlab-postgresql.service...1148vm-test-run-nixbot-gitlab> gitlab # [ 10.290409] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1351]: mkdir -p /var/gitlab/state/home/.ssh: OK1149vm-test-run-nixbot-gitlab> gitlab # [ 10.293822] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1351]: chmod 700 /var/gitlab/state/home/.ssh: OK1150vm-test-run-nixbot-gitlab> gitlab # [ 10.295986] gitlab-postgresql-start[1368]: CREATE DATABASE1151vm-test-run-nixbot-gitlab> gitlab # [ 10.342219] systemd[1]: Finished gitlab-config.service.1152vm-test-run-nixbot-gitlab> gitlab # [ 10.344957] gitlab-postgresql-start[1379]: CREATE EXTENSION1153vm-test-run-nixbot-gitlab> gitlab # [ 10.346795] systemd[1]: Started gitaly.service.1154vm-test-run-nixbot-gitlab> gitlab # [ 10.350870] systemd[1]: Starting gitlab-db-config.service...1155vm-test-run-nixbot-gitlab> gitlab # [ 10.391335] gitlab-postgresql-start[1388]: CREATE EXTENSION1156vm-test-run-nixbot-gitlab> gitlab # [ 10.397401] systemd[1]: Finished gitlab-postgresql.service.1157vm-test-run-nixbot-gitlab> gitlab # [ 10.951977] gitaly[1386]: time="2026-06-17T14:51:35.514Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13861158vm-test-run-nixbot-gitlab> gitlab # [ 10.954554] gitaly[1386]: time="2026-06-17T14:51:35.515Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13861159vm-test-run-nixbot-gitlab> gitlab # [ 10.956674] gitaly[1386]: time="2026-06-17T14:51:35.515Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13861160vm-test-run-nixbot-gitlab> gitlab # [ 10.959465] gitaly[1386]: time="2026-06-17T14:51:35.515Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13861161vm-test-run-nixbot-gitlab> gitlab # [ 10.961677] gitaly[1386]: time="2026-06-17T14:51:35.515Z" level=info msg="Starting Gitaly" pid=1386 version=18.11.41162vm-test-run-nixbot-gitlab> gitlab # [ 11.174428] gitaly[1386]: time="2026-06-17T14:51:35.737Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=220 pid=13861163vm-test-run-nixbot-gitlab> gitlab # [ 11.176112] gitaly[1386]: time="2026-06-17T14:51:35.737Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13861164vm-test-run-nixbot-gitlab> gitlab # [ 11.176902] gitaly[1386]: time="2026-06-17T14:51:35.738Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13861165vm-test-run-nixbot-gitlab> gitlab # [ 11.178383] gitaly[1386]: time="2026-06-17T14:51:35.738Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1386/git-exec-1801709716.d/git pid=13861166vm-test-run-nixbot-gitlab> gitlab # [ 11.193044] gitaly[1386]: time="2026-06-17T14:51:35.756Z" level=info msg="finished detecting git version" duration_ms=17 pid=13861167vm-test-run-nixbot-gitlab> gitlab # [ 11.194430] gitaly[1386]: time="2026-06-17T14:51:35.756Z" level=info msg="using Git version" pid=1386 version=2.53.g1168vm-test-run-nixbot-gitlab> gitlab # [ 11.200103] gitaly[1386]: time="2026-06-17T14:51:35.763Z" level=info msg="clearing disk cache object folder" pid=1386 storage=default1169vm-test-run-nixbot-gitlab> gitlab # [ 11.203875] gitaly[1386]: time="2026-06-17T14:51:35.767Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2456303950 pid=1386 storage=default1170vm-test-run-nixbot-gitlab> gitlab # [ 11.205193] gitaly[1386]: time="2026-06-17T14:51:35.767Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2456303950 pid=1386 storage=default1171vm-test-run-nixbot-gitlab> gitlab # [ 11.206367] gitaly[1386]: time="2026-06-17T14:51:35.767Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13861172vm-test-run-nixbot-gitlab> gitlab # [ 11.207173] gitaly[1386]: time="2026-06-17T14:51:35.767Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13861173vm-test-run-nixbot-gitlab> gitlab # [ 11.207909] gitaly[1386]: time="2026-06-17T14:51:35.767Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2456303950 clear_duration_ms=0 pid=1386 storage=default1174vm-test-run-nixbot-gitlab> gitlab # [ 11.302358] gitaly[1386]: time="2026-06-17T14:51:35.864Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13861175vm-test-run-nixbot-gitlab> gitlab # [ 11.305485] gitaly[1386]: time="2026-06-17T14:51:35.865Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1386 schema=unix1176vm-test-run-nixbot-gitlab> gitlab # [ 11.307938] gitaly[1386]: time="2026-06-17T14:51:35.865Z" level=info msg="counting repositories" pid=13861177vm-test-run-nixbot-gitlab> gitlab # [ 11.310959] gitaly[1386]: time="2026-06-17T14:51:35.865Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1386/sock.d/intern pid=1386 schema=unix1178vm-test-run-nixbot-gitlab> gitlab # [ 11.315106] gitaly[1386]: time="2026-06-17T14:51:35.865Z" level=info msg="starting to count repositories" pid=1386 storage_path=/var/gitlab/state/repositories1179vm-test-run-nixbot-gitlab> gitlab # [ 11.317590] gitaly[1386]: time="2026-06-17T14:51:35.865Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1386 storage_path=/var/gitlab/state/repositories1180vm-test-run-nixbot-gitlab> gitlab # [ 11.320426] gitaly[1386]: time="2026-06-17T14:51:35.865Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13861181vm-test-run-nixbot-gitlab> gitlab # [ 11.322704] gitaly[1386]: time="2026-06-17T14:51:35.865Z" level=info msg="finished tempdir cleaner walk" pid=1386 storage=default time_ms=01182vm-test-run-nixbot-gitlab> gitlab # [ 11.324988] gitaly[1386]: time="2026-06-17T14:51:35.870Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13861183vm-test-run-nixbot-gitlab> gitlab # [ 11.326676] gitaly[1386]: time="2026-06-17T14:51:35.870Z" level=info msg="Started Gitaly" duration_ms=354 pid=13861184vm-test-run-nixbot-gitlab> gitlab # [ 11.327786] gitaly[1386]: time="2026-06-17T14:51:35.870Z" level=info msg="maintenance: daily scheduled" pid=1386 scheduled="2026-06-18 12:00:00 +0000 UTC"1185vm-test-run-nixbot-gitlab> gitlab # [ 11.498709] dhcpcd[1235]: eth0: soliciting an IPv6 router1186vm-test-run-nixbot-gitlab> gitlab # [ 11.502345] dhcpcd[1235]: eth0: Router Advertisement from fe80::21187vm-test-run-nixbot-gitlab> gitlab # [ 11.505528] dhcpcd[1235]: eth0: adding address fec0::5054:ff:fe12:3456/641188vm-test-run-nixbot-gitlab> gitlab # [ 11.507161] dhcpcd[1235]: eth0: adding route to fec0::/641189vm-test-run-nixbot-gitlab> gitlab # [ 11.508209] dhcpcd[1235]: eth0: adding default route via fe80::21190vm-test-run-nixbot-gitlab> gitlab # [ 11.705512] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1391]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1191vm-test-run-nixbot-gitlab> gitlab # [ 11.707759] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1391]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1192vm-test-run-nixbot-gitlab> gitlab # [ 11.751906] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1391]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1193vm-test-run-nixbot-gitlab> gitlab # [ 13.220160] gitaly[1386]: time="2026-06-17T14:51:37.783Z" level=info msg="License database preloaded" duration_ms=2267 pid=13861194vm-test-run-nixbot-gitlab> gitlab # [ 14.903725] dhcpcd[1235]: eth0: leased 10.0.2.15 for 86400 seconds1195vm-test-run-nixbot-gitlab> gitlab # [ 14.907505] dhcpcd[1235]: eth0: adding route to 10.0.2.0/241196vm-test-run-nixbot-gitlab> gitlab # [ 14.910209] dhcpcd[1235]: eth0: adding default route via 10.0.2.21197vm-test-run-nixbot-gitlab> gitlab # [ 15.047685] systemd[1]: Started DHCP Client.1198vm-test-run-nixbot-gitlab> gitlab # [ 15.051874] systemd[1]: Reached target Network is Online.1199vm-test-run-nixbot-gitlab> gitlab # [ 44.212900] postgres[1769]: [1769] ERROR: relation "feature_gates" does not exist at character 5231200vm-test-run-nixbot-gitlab> gitlab # [ 44.215393] postgres[1769]: [1769] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1201vm-test-run-nixbot-gitlab> gitlab # [ 44.217200] postgres[1769]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1202vm-test-run-nixbot-gitlab> gitlab # [ 44.218893] postgres[1769]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1203vm-test-run-nixbot-gitlab> gitlab # [ 44.220972] postgres[1769]: attidentity AS identity,1204vm-test-run-nixbot-gitlab> gitlab # [ 44.222573] postgres[1769]: attgenerated as attgenerated1205vm-test-run-nixbot-gitlab> gitlab # [ 44.224490] postgres[1769]: FROM pg_attribute a1206vm-test-run-nixbot-gitlab> gitlab # [ 44.226918] postgres[1769]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1207vm-test-run-nixbot-gitlab> gitlab # [ 44.229748] postgres[1769]: LEFT JOIN pg_type t ON a.atttypid = t.oid1208vm-test-run-nixbot-gitlab> gitlab # [ 44.232460] postgres[1769]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1209vm-test-run-nixbot-gitlab> gitlab # [ 44.235128] postgres[1769]: WHERE a.attrelid = '"feature_gates"'::regclass1210vm-test-run-nixbot-gitlab> gitlab # [ 44.238245] postgres[1769]: AND a.attnum > 0 AND NOT a.attisdropped1211vm-test-run-nixbot-gitlab> gitlab # [ 44.240886] postgres[1769]: ORDER BY a.attnum1212vm-test-run-nixbot-gitlab> gitlab # [ 44.243271] postgres[1769]:1213vm-test-run-nixbot-gitlab> gitlab # [ 44.475531] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1391]: W, [2026-06-17T14:52:09.037689 #1391] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1214vm-test-run-nixbot-gitlab> gitlab # [ 48.920427] gitlab-workhorse[1289]: time="2026-06-17T14:52:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1215vm-test-run-nixbot-gitlab> gitlab # [ 50.543429] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1833]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1216vm-test-run-nixbot-gitlab> gitlab # [ 50.547707] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1833]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1217vm-test-run-nixbot-gitlab> gitlab # [ 58.929835] gitlab-workhorse[1289]: time="2026-06-17T14:52:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1218vm-test-run-nixbot-gitlab> gitlab # [ 67.928139] postgres[1908]: [1908] ERROR: relation "table_without_models" does not exist1219vm-test-run-nixbot-gitlab> gitlab # [ 67.932220] postgres[1908]: [1908] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1220vm-test-run-nixbot-gitlab> gitlab # [ 68.939387] gitlab-workhorse[1289]: time="2026-06-17T14:52:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1221vm-test-run-nixbot-gitlab> gitlab # [ 78.949117] gitlab-workhorse[1289]: time="2026-06-17T14:52:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1222vm-test-run-nixbot-gitlab> gitlab # [ 83.826189] systemd[1]: Finished gitlab-db-config.service.1223vm-test-run-nixbot-gitlab> gitlab # [ 83.832121] systemd[1]: Started gitlab-sidekiq.service.1224vm-test-run-nixbot-gitlab> gitlab # [ 83.835555] systemd[1]: Starting gitlab.service...1225vm-test-run-nixbot-gitlab> gitlab # [ 84.206173] bundle[2120]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1226vm-test-run-nixbot-gitlab> gitlab # [ 84.214790] bundle[2120]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1227vm-test-run-nixbot-gitlab> gitlab # [ 84.223137] bundle[2120]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1228vm-test-run-nixbot-gitlab> gitlab # [ 87.267542] sidekiq-cluster[2119]: {"severity":"INFO","time":"2026-06-17T14:52:51.812Z","message":"Starting cluster with 1 processes"}1229vm-test-run-nixbot-gitlab> gitlab # [ 87.367786] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.907Z","pid":2120,"message":"* Enabling systemd notification integration"}1230vm-test-run-nixbot-gitlab> gitlab # [ 87.368967] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.930Z","pid":2120,"message":"Puma starting in cluster mode..."}1231vm-test-run-nixbot-gitlab> gitlab # [ 87.370334] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.930Z","pid":2120,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1232vm-test-run-nixbot-gitlab> gitlab # [ 87.372133] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.930Z","pid":2120,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1233vm-test-run-nixbot-gitlab> gitlab # [ 87.373482] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.930Z","pid":2120,"message":"* Min threads: 0"}1234vm-test-run-nixbot-gitlab> gitlab # [ 87.374844] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.930Z","pid":2120,"message":"* Max threads: 4"}1235vm-test-run-nixbot-gitlab> gitlab # [ 87.376430] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.930Z","pid":2120,"message":"* Environment: production"}1236vm-test-run-nixbot-gitlab> gitlab # [ 87.377814] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.931Z","pid":2120,"message":"* Master PID: 2120"}1237vm-test-run-nixbot-gitlab> gitlab # [ 87.379319] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.931Z","pid":2120,"message":"* Workers: 2"}1238vm-test-run-nixbot-gitlab> gitlab # [ 87.380577] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.931Z","pid":2120,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1239vm-test-run-nixbot-gitlab> gitlab # [ 87.382265] bundle[2120]: {"timestamp":"2026-06-17T14:52:51.931Z","pid":2120,"message":"* Preloading application"}1240vm-test-run-nixbot-gitlab> gitlab # [ 88.959685] gitlab-workhorse[1289]: time="2026-06-17T14:52:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1241vm-test-run-nixbot-gitlab> gitlab # [ 98.961775] gitlab-workhorse[1289]: time="2026-06-17T14:53:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1242vm-test-run-nixbot-gitlab> gitlab # [ 108.963273] gitlab-workhorse[1289]: time="2026-06-17T14:53:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1243vm-test-run-nixbot-gitlab> gitlab # [ 112.655852] bundle[2120]: W, [2026-06-17T14:53:17.218068 #2120] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1244vm-test-run-nixbot-gitlab> gitlab # [ 116.190720] sidekiq-cluster[2150]: 2026-06-17T14:53:20.753Z pid=2150 tid=u0e INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2150, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430096384}1245vm-test-run-nixbot-gitlab> gitlab # [ 116.382617] sidekiq-cluster[2150]: W, [2026-06-17T14:53:20.945714 #2150] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1246vm-test-run-nixbot-gitlab> gitlab # [ 116.425220] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:20.988Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1247vm-test-run-nixbot-gitlab> gitlab # [ 118.448858] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.011Z","message":"GitLab reliable fetch activated!"}1248vm-test-run-nixbot-gitlab> gitlab # [ 118.450641] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.011Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1249vm-test-run-nixbot-gitlab> gitlab # [ 118.475614] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.038Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1250vm-test-run-nixbot-gitlab> gitlab # [ 118.478811] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.041Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1251vm-test-run-nixbot-gitlab> gitlab # [ 118.482616] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.045Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1252vm-test-run-nixbot-gitlab> gitlab # [ 118.487170] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.050Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1253vm-test-run-nixbot-gitlab> gitlab # [ 118.490914] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.054Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [ 118.494384] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.057Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [ 118.500916] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.063Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [ 118.506782] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.069Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [ 118.515554] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.078Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [ 118.524597] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.087Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [ 118.533867] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.096Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [ 118.544842] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.107Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [ 118.553612] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.116Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [ 118.562608] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.125Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [ 118.571867] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.134Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [ 118.582209] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.145Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [ 118.592817] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.155Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [ 118.603548] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.166Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [ 118.613526] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.176Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [ 118.623651] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.186Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [ 118.634210] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.197Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [ 118.644636] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.207Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [ 118.654677] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.217Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [ 118.664231] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.227Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [ 118.673522] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.236Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [ 118.684108] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.247Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [ 118.694545] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.257Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [ 118.704541] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.267Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [ 118.713950] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.277Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [ 118.724211] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.287Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [ 118.734223] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.297Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [ 118.743230] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.306Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [ 118.753648] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.316Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [ 118.762872] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.325Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [ 118.772920] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.336Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [ 118.781946] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.345Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1285vm-test-run-nixbot-gitlab> gitlab # [ 118.790981] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.354Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1286vm-test-run-nixbot-gitlab> gitlab # [ 118.800213] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.363Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1287vm-test-run-nixbot-gitlab> gitlab # [ 118.809964] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.373Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1288vm-test-run-nixbot-gitlab> gitlab # [ 118.818904] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.382Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1289vm-test-run-nixbot-gitlab> gitlab # [ 118.827853] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.390Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1290vm-test-run-nixbot-gitlab> gitlab # [ 118.837191] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.400Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1291vm-test-run-nixbot-gitlab> gitlab # [ 118.846223] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.409Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1292vm-test-run-nixbot-gitlab> gitlab # [ 118.856825] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.419Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1293vm-test-run-nixbot-gitlab> gitlab # [ 118.866261] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.429Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1294vm-test-run-nixbot-gitlab> gitlab # [ 118.876562] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.439Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1295vm-test-run-nixbot-gitlab> gitlab # [ 118.885613] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.448Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1296vm-test-run-nixbot-gitlab> gitlab # [ 118.894639] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.457Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1297vm-test-run-nixbot-gitlab> gitlab # [ 118.903900] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.467Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1298vm-test-run-nixbot-gitlab> gitlab # [ 118.913640] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.476Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1299vm-test-run-nixbot-gitlab> gitlab # [ 118.922850] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.485Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1300vm-test-run-nixbot-gitlab> gitlab # [ 118.932633] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.495Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1301vm-test-run-nixbot-gitlab> gitlab # [ 118.942677] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.505Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1302vm-test-run-nixbot-gitlab> gitlab # [ 118.964779] gitlab-workhorse[1289]: time="2026-06-17T14:53:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1303vm-test-run-nixbot-gitlab> gitlab # [ 118.966635] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.528Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1304vm-test-run-nixbot-gitlab> gitlab # [ 118.979190] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.542Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1305vm-test-run-nixbot-gitlab> gitlab # [ 118.987621] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.550Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1306vm-test-run-nixbot-gitlab> gitlab # [ 118.996207] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.559Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1307vm-test-run-nixbot-gitlab> gitlab # [ 119.006261] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.569Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1308vm-test-run-nixbot-gitlab> gitlab # [ 119.015636] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.578Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1309vm-test-run-nixbot-gitlab> gitlab # [ 119.024752] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.587Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1310vm-test-run-nixbot-gitlab> gitlab # [ 119.033650] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.596Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1311vm-test-run-nixbot-gitlab> gitlab # [ 119.042253] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.605Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1312vm-test-run-nixbot-gitlab> gitlab # [ 119.050213] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.613Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1313vm-test-run-nixbot-gitlab> gitlab # [ 119.059184] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.622Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1314vm-test-run-nixbot-gitlab> gitlab # [ 119.068528] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.631Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1315vm-test-run-nixbot-gitlab> gitlab # [ 119.078351] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.641Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1316vm-test-run-nixbot-gitlab> gitlab # [ 119.112673] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.674Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1317vm-test-run-nixbot-gitlab> gitlab # [ 119.118899] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.682Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1318vm-test-run-nixbot-gitlab> gitlab # [ 119.126197] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.689Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1319vm-test-run-nixbot-gitlab> gitlab # [ 119.133187] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.696Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1320vm-test-run-nixbot-gitlab> gitlab # [ 119.140241] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.703Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1321vm-test-run-nixbot-gitlab> gitlab # [ 119.147907] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.711Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1322vm-test-run-nixbot-gitlab> gitlab # [ 119.154660] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.717Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1323vm-test-run-nixbot-gitlab> gitlab # [ 119.166589] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.729Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1324vm-test-run-nixbot-gitlab> gitlab # [ 119.175595] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.738Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1325vm-test-run-nixbot-gitlab> gitlab # [ 119.183987] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.747Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1326vm-test-run-nixbot-gitlab> gitlab # [ 119.194779] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.757Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1327vm-test-run-nixbot-gitlab> gitlab # [ 119.204799] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.767Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1328vm-test-run-nixbot-gitlab> gitlab # [ 119.214364] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.777Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1329vm-test-run-nixbot-gitlab> gitlab # [ 119.223707] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.786Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1330vm-test-run-nixbot-gitlab> gitlab # [ 119.233606] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.796Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1331vm-test-run-nixbot-gitlab> gitlab # [ 119.243832] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.806Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1332vm-test-run-nixbot-gitlab> gitlab # [ 119.254223] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.817Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1333vm-test-run-nixbot-gitlab> gitlab # [ 119.264639] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.827Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1334vm-test-run-nixbot-gitlab> gitlab # [ 119.276129] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.839Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1335vm-test-run-nixbot-gitlab> gitlab # [ 119.285559] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.848Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1336vm-test-run-nixbot-gitlab> gitlab # [ 119.296215] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.859Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1337vm-test-run-nixbot-gitlab> gitlab # [ 119.306390] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.869Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1338vm-test-run-nixbot-gitlab> gitlab # [ 119.316595] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.879Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1339vm-test-run-nixbot-gitlab> gitlab # [ 119.325647] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.888Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1340vm-test-run-nixbot-gitlab> gitlab # [ 119.335669] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.898Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1341vm-test-run-nixbot-gitlab> gitlab # [ 119.344630] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.907Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1342vm-test-run-nixbot-gitlab> gitlab # [ 119.353234] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.916Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1343vm-test-run-nixbot-gitlab> gitlab # [ 119.363205] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.926Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1344vm-test-run-nixbot-gitlab> gitlab # [ 119.372347] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.935Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1345vm-test-run-nixbot-gitlab> gitlab # [ 119.383604] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.946Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1346vm-test-run-nixbot-gitlab> gitlab # [ 119.394180] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.957Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1347vm-test-run-nixbot-gitlab> gitlab # [ 119.403602] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.966Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1348vm-test-run-nixbot-gitlab> gitlab # [ 119.412612] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:23.975Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1349vm-test-run-nixbot-gitlab> gitlab # [ 119.438489] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.001Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1350vm-test-run-nixbot-gitlab> gitlab # [ 119.446208] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.009Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1351vm-test-run-nixbot-gitlab> gitlab # [ 119.458898] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.021Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1352vm-test-run-nixbot-gitlab> gitlab # [ 119.469827] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.032Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1353vm-test-run-nixbot-gitlab> gitlab # [ 119.479523] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.042Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1354vm-test-run-nixbot-gitlab> gitlab # [ 119.486276] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.049Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1355vm-test-run-nixbot-gitlab> gitlab # [ 119.493734] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.056Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1356vm-test-run-nixbot-gitlab> gitlab # [ 119.497459] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.060Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1357vm-test-run-nixbot-gitlab> gitlab # [ 119.503380] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.066Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1358vm-test-run-nixbot-gitlab> gitlab # [ 119.512068] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.075Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1359vm-test-run-nixbot-gitlab> gitlab # [ 119.518709] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:24.080Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1360vm-test-run-nixbot-gitlab> gitlab # [ 128.967567] gitlab-workhorse[1289]: time="2026-06-17T14:53:33Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1361vm-test-run-nixbot-gitlab> gitlab # [ 138.974719] gitlab-workhorse[1289]: time="2026-06-17T14:53:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1362vm-test-run-nixbot-gitlab> gitlab # [ 142.911774] bundle[2120]: {"timestamp":"2026-06-17T14:53:47.473Z","pid":2120,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1363vm-test-run-nixbot-gitlab> gitlab # [ 142.920134] bundle[2120]: {"timestamp":"2026-06-17T14:53:47.474Z","pid":2120,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1364vm-test-run-nixbot-gitlab> gitlab # [ 142.927260] bundle[2120]: {"timestamp":"2026-06-17T14:53:47.474Z","pid":2120,"message":"! #\u003cThread:0x000078996699b308 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1365vm-test-run-nixbot-gitlab> gitlab # [ 142.951438] bundle[2120]: {"timestamp":"2026-06-17T14:53:47.474Z","pid":2120,"message":"! #\u003cThread:0x0000789966bb7c40 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1366vm-test-run-nixbot-gitlab> gitlab # [ 142.967667] bundle[2120]: {"timestamp":"2026-06-17T14:53:47.474Z","pid":2120,"message":"Use Ctrl-C to stop"}1367vm-test-run-nixbot-gitlab> gitlab # [ 142.972310] bundle[2120]: {"timestamp":"2026-06-17T14:53:47.474Z","pid":2120,"message":"Sending status to systemd every 1.0 sec"}1368vm-test-run-nixbot-gitlab> gitlab # [ 143.353676] bundle[2120]: {"timestamp":"2026-06-17T14:53:47.916Z","pid":2120,"message":"- Worker 0 (PID: 2678) booted in 0.41s, phase: 0"}1369vm-test-run-nixbot-gitlab> gitlab # [ 143.373507] bundle[2120]: {"timestamp":"2026-06-17T14:53:47.936Z","pid":2120,"message":"- Worker 1 (PID: 2681) booted in 0.4s, phase: 0"}1370vm-test-run-nixbot-gitlab> gitlab # [ 143.382053] systemd[1]: Started gitlab.service.1371vm-test-run-nixbot-gitlab> gitlab # [ 143.384074] systemd[1]: Reached target Common target for all GitLab services..1372vm-test-run-nixbot-gitlab> gitlab # [ 143.388264] systemd[1]: Starting setup-gitlab.service...1373vm-test-run-nixbot-gitlab> gitlab # [ 143.409938] setup-gitlab-start[2717]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1374vm-test-run-nixbot-gitlab> gitlab # [ 143.420886] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:47.983Z","message":"Booted Rails 7.2.3 application in production environment"}1375vm-test-run-nixbot-gitlab> gitlab # [ 143.422146] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:47.984Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1376vm-test-run-nixbot-gitlab> gitlab # [ 143.424207] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:47.984Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1377vm-test-run-nixbot-gitlab> gitlab # [ 143.426862] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:47.984Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1378vm-test-run-nixbot-gitlab> gitlab # [ 143.450603] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:48.013Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1379vm-test-run-nixbot-gitlab> gitlab # [ 145.032788] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:49.595Z","message":"Cleaning working queues"}1380vm-test-run-nixbot-gitlab> gitlab # [ 145.045607] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:49.608Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c3feb85307b0f8ddb10fa683","created_at":"2026-06-17T14:52:47.804Z","trace_propagation_headers":{"sentry-trace":"b6ce51f1e9524acab72e545d12a77f62-61a351a6ff3542d4","baggage":"sentry-trace_id=b6ce51f1e9524acab72e545d12a77f62,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"93f7b0ffb68fa4b968e1a81b8394985e","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-17T14:52:47.818Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c3feb85307b0f8ddb10fa683: start","job_status":"start","queue_duration_s":61.789782,"scheduling_latency_s":61.789877}1381vm-test-run-nixbot-gitlab> gitlab # [ 145.264848] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:53:49.827Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c3feb85307b0f8ddb10fa683","created_at":"2026-06-17T14:52:47.804Z","trace_propagation_headers":{"sentry-trace":"b6ce51f1e9524acab72e545d12a77f62-61a351a6ff3542d4","baggage":"sentry-trace_id=b6ce51f1e9524acab72e545d12a77f62,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"93f7b0ffb68fa4b968e1a81b8394985e","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-17T14:52:47.818Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c3feb85307b0f8ddb10fa683: done: 0.219389 sec","job_status":"done","queue_duration_s":61.789782,"scheduling_latency_s":61.789877,"redis_calls":26,"redis_duration_s":0.011213999999999998,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.004912,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004029,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002273,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.107592725,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.219389,"completed_at":"2026-06-17T14:53:49.827Z","load_balancing_strategy":"primary","db_duration_s":0.055888,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1382vm-test-run-nixbot-gitlab> gitlab # [ 145.897670] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:53:50 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 24341383vm-test-run-nixbot-gitlab> gitlab # [ 145.911304] setup-gitlab-start[2743]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1384vm-test-run-nixbot-gitlab> gitlab # [ 145.914189] setup-gitlab-start[2744]: ++ jq -r .access_token1385vm-test-run-nixbot-gitlab> gitlab # [ 147.295631] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:53:51 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 13491386vm-test-run-nixbot-gitlab> gitlab # [ 147.300759] setup-gitlab-start[2717]: + BEARER=69f526118babb9072f36fc6d073f255f60d20abf9dd87392e1efb81a4e7972b61387vm-test-run-nixbot-gitlab> gitlab # [ 147.304240] setup-gitlab-start[2717]: + AUTH='Authorization: Bearer 69f526118babb9072f36fc6d073f255f60d20abf9dd87392e1efb81a4e7972b6'1388vm-test-run-nixbot-gitlab> gitlab # [ 147.306978] setup-gitlab-start[2717]: + curl -fs -X PUT -H 'Authorization: Bearer 69f526118babb9072f36fc6d073f255f60d20abf9dd87392e1efb81a4e7972b6' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1389vm-test-run-nixbot-gitlab> gitlab # [ 148.088532] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:53:52 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 7671390vm-test-run-nixbot-gitlab> gitlab # [ 148.092119] setup-gitlab-start[2717]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1391vm-test-run-nixbot-gitlab> gitlab # [ 148.109251] runuser[2766]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1392vm-test-run-nixbot-gitlab> gitlab # [ 148.505422] setup-gitlab-start[2767]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1393vm-test-run-nixbot-gitlab> gitlab # [ 148.508168] setup-gitlab-start[2767]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1394vm-test-run-nixbot-gitlab> gitlab # [ 148.510961] setup-gitlab-start[2767]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1395vm-test-run-nixbot-gitlab> gitlab # [ 149.097296] setup-gitlab-start[2767]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1396vm-test-run-nixbot-gitlab> gitlab # [ 149.100234] setup-gitlab-start[2767]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1397vm-test-run-nixbot-gitlab> gitlab # [ 149.102131] setup-gitlab-start[2767]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1398vm-test-run-nixbot-gitlab> gitlab # [ 157.944877] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.506Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e061591e5a60b06f65d24193","created_at":"2026-06-17T14:54:02.468Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"546bb20de9fa930d1fbff3bc58c79ba0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:02.503Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e061591e5a60b06f65d24193: start","job_status":"start","queue_duration_s":0.003559,"scheduling_latency_s":0.003609}1399vm-test-run-nixbot-gitlab> gitlab # [ 158.014547] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.577Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1400vm-test-run-nixbot-gitlab> gitlab # [ 158.040165] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.603Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e061591e5a60b06f65d24193","created_at":"2026-06-17T14:54:02.468Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"546bb20de9fa930d1fbff3bc58c79ba0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:02.503Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e061591e5a60b06f65d24193: done: 0.09659 sec","job_status":"done","queue_duration_s":0.003559,"scheduling_latency_s":0.003609,"redis_calls":10,"redis_duration_s":0.019659,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00292,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.013502554,"duration_s":0.09659,"completed_at":"2026-06-17T14:54:02.603Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401vm-test-run-nixbot-gitlab> gitlab # [ 158.129336] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.692Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"097d85d0d8e889d24970a7d4","created_at":"2026-06-17T14:54:02.527Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2accf15aa2a626f191cd10bfef1a057","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-17T14:54:02.554Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-097d85d0d8e889d24970a7d4: start","job_status":"start","queue_duration_s":0.137914,"scheduling_latency_s":0.137956}1402vm-test-run-nixbot-gitlab> gitlab # [ 158.183608] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.746Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1403vm-test-run-nixbot-gitlab> gitlab # [ 158.211806] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.774Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"097d85d0d8e889d24970a7d4","created_at":"2026-06-17T14:54:02.527Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2accf15aa2a626f191cd10bfef1a057","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-17T14:54:02.554Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-097d85d0d8e889d24970a7d4: done: 0.081894 sec","job_status":"done","queue_duration_s":0.137914,"scheduling_latency_s":0.137956,"redis_calls":10,"redis_duration_s":0.020567,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.008766,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009179,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002622,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001653003,"duration_s":0.081894,"completed_at":"2026-06-17T14:54:02.773Z","load_balancing_strategy":"primary","db_duration_s":0.002038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404vm-test-run-nixbot-gitlab> gitlab # [ 158.248684] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.784Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4f3208b6ed26a34d5670db95","created_at":"2026-06-17T14:54:02.567Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"755c1ad42657e8e7b8ebf7bef352452c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-17T14:54:02.585Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4f3208b6ed26a34d5670db95: start","job_status":"start","queue_duration_s":0.198782,"scheduling_latency_s":0.198827}1405vm-test-run-nixbot-gitlab> gitlab # [ 158.262722] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.825Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1406vm-test-run-nixbot-gitlab> gitlab # [ 158.276734] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4f3208b6ed26a34d5670db95","created_at":"2026-06-17T14:54:02.567Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"755c1ad42657e8e7b8ebf7bef352452c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-17T14:54:02.585Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4f3208b6ed26a34d5670db95: done: 0.054908 sec","job_status":"done","queue_duration_s":0.198782,"scheduling_latency_s":0.198827,"redis_calls":10,"redis_duration_s":0.006554,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001123,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00403,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000776077,"duration_s":0.054908,"completed_at":"2026-06-17T14:54:02.839Z","load_balancing_strategy":"primary","db_duration_s":0.001042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407vm-test-run-nixbot-gitlab> gitlab # [ 158.298562] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.846Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"04aa34e1a59570046a846f37","created_at":"2026-06-17T14:54:02.595Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d55a15a5dbe27b4948c7d781d1294f1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:02.691Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-04aa34e1a59570046a846f37: start","job_status":"start","queue_duration_s":0.154389,"scheduling_latency_s":0.154413}1408vm-test-run-nixbot-gitlab> gitlab # [ 158.321391] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.862Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1409vm-test-run-nixbot-gitlab> gitlab # [ 158.327107] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.871Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"04aa34e1a59570046a846f37","created_at":"2026-06-17T14:54:02.595Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d55a15a5dbe27b4948c7d781d1294f1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:02.691Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-04aa34e1a59570046a846f37: done: 0.025332 sec","job_status":"done","queue_duration_s":0.154389,"scheduling_latency_s":0.154413,"redis_calls":10,"redis_duration_s":0.007559,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001068,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001145397,"duration_s":0.025332,"completed_at":"2026-06-17T14:54:02.871Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410vm-test-run-nixbot-gitlab> gitlab # [ 158.461203] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:02.874Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ef595e9402e5a0332e329826","created_at":"2026-06-17T14:54:02.703Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:02.724Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ef595e9402e5a0332e329826: start","job_status":"start","queue_duration_s":0.149918,"scheduling_latency_s":0.14994}1411vm-test-run-nixbot-gitlab> gitlab # [ 158.487569] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ef595e9402e5a0332e329826","created_at":"2026-06-17T14:54:02.703Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:02.724Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ef595e9402e5a0332e329826: done: 0.175327 sec","job_status":"done","queue_duration_s":0.149918,"scheduling_latency_s":0.14994,"redis_calls":71,"redis_duration_s":0.022761,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003352,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000939,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.016996,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001474,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.135691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000379936,"duration_s":0.175327,"completed_at":"2026-06-17T14:54:03.050Z","load_balancing_strategy":"primary","db_duration_s":0.004321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412vm-test-run-nixbot-gitlab> gitlab # [ 158.526110] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.060Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"03b433c8a2e414f08b178952","created_at":"2026-06-17T14:54:02.747Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5375cf8afc36ca9cc272d8b430e7ff85","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:02.777Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-03b433c8a2e414f08b178952: start","job_status":"start","queue_duration_s":0.282915,"scheduling_latency_s":0.282958}1413vm-test-run-nixbot-gitlab> gitlab # [ 158.643768] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.206Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"03b433c8a2e414f08b178952","created_at":"2026-06-17T14:54:02.747Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5375cf8afc36ca9cc272d8b430e7ff85","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:02.777Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-03b433c8a2e414f08b178952: done: 0.146123 sec","job_status":"done","queue_duration_s":0.282915,"scheduling_latency_s":0.282958,"redis_calls":12,"redis_duration_s":0.063666,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00117,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.058711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003785,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001471416,"exclusive_lock_hold_duration_s":0.011432739,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.146123,"completed_at":"2026-06-17T14:54:03.206Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414vm-test-run-nixbot-gitlab> gitlab # [ 158.717982] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3666affd5c401e5ecff16365","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3666affd5c401e5ecff16365: start","job_status":"start","queue_duration_s":0.250435,"scheduling_latency_s":0.250479}1415vm-test-run-nixbot-gitlab> gitlab # [ 158.790609] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3666affd5c401e5ecff16365","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3666affd5c401e5ecff16365: done: 0.072532 sec","job_status":"done","queue_duration_s":0.250435,"scheduling_latency_s":0.250479,"redis_calls":16,"redis_duration_s":0.018590000000000002,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002677,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009369,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001359391,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.072532,"completed_at":"2026-06-17T14:54:03.353Z","load_balancing_strategy":"primary","db_duration_s":0.000957,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416vm-test-run-nixbot-gitlab> gitlab # [ 158.863270] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36f06acfcb0d395076a9b999","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36f06acfcb0d395076a9b999: start","job_status":"start","queue_duration_s":0.395369,"scheduling_latency_s":0.395447}1417vm-test-run-nixbot-gitlab> gitlab # [ 158.951944] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36f06acfcb0d395076a9b999","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36f06acfcb0d395076a9b999: done: 0.087922 sec","job_status":"done","queue_duration_s":0.395369,"scheduling_latency_s":0.395447,"redis_calls":13,"redis_duration_s":0.036716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.019748,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.016968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.087922,"completed_at":"2026-06-17T14:54:03.513Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418vm-test-run-nixbot-gitlab> gitlab # [ 159.030270] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"949ed43dd36f23bd764942e0","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-949ed43dd36f23bd764942e0: start","job_status":"start","queue_duration_s":0.562432,"scheduling_latency_s":0.562479}1419vm-test-run-nixbot-gitlab> gitlab # [ 159.081191] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"949ed43dd36f23bd764942e0","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-949ed43dd36f23bd764942e0: done: 0.050929 sec","job_status":"done","queue_duration_s":0.562432,"scheduling_latency_s":0.562479,"redis_calls":13,"redis_duration_s":0.010069999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007434,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.050929,"completed_at":"2026-06-17T14:54:03.643Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420vm-test-run-nixbot-gitlab> gitlab # [ 159.083859] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd936700739eb22b8454ee0f","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd936700739eb22b8454ee0f: start","job_status":"start","queue_duration_s":0.615262,"scheduling_latency_s":0.61529}1421vm-test-run-nixbot-gitlab> gitlab # [ 159.239614] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd936700739eb22b8454ee0f","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd936700739eb22b8454ee0f: done: 0.15651 sec","job_status":"done","queue_duration_s":0.615262,"scheduling_latency_s":0.61529,"redis_calls":13,"redis_duration_s":0.011954,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.15651,"completed_at":"2026-06-17T14:54:03.802Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422vm-test-run-nixbot-gitlab> gitlab # [ 159.244657] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e41d205edf5d98a0abe20de","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e41d205edf5d98a0abe20de: start","job_status":"start","queue_duration_s":0.773423,"scheduling_latency_s":0.773447}1423vm-test-run-nixbot-gitlab> gitlab # [ 159.263976] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e41d205edf5d98a0abe20de","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e41d205edf5d98a0abe20de: done: 0.015979 sec","job_status":"done","queue_duration_s":0.773423,"scheduling_latency_s":0.773447,"redis_calls":13,"redis_duration_s":0.0030440000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015979,"completed_at":"2026-06-17T14:54:03.819Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424vm-test-run-nixbot-gitlab> gitlab # [ 159.329959] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc8e29c2ec75049f54c65ea2","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc8e29c2ec75049f54c65ea2: start","job_status":"start","queue_duration_s":0.790853,"scheduling_latency_s":0.790874}1425vm-test-run-nixbot-gitlab> gitlab # [ 159.350114] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc8e29c2ec75049f54c65ea2","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc8e29c2ec75049f54c65ea2: done: 0.015913 sec","job_status":"done","queue_duration_s":0.790853,"scheduling_latency_s":0.790874,"redis_calls":13,"redis_duration_s":0.003084,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015913,"completed_at":"2026-06-17T14:54:03.837Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426vm-test-run-nixbot-gitlab> gitlab # [ 159.470601] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02e8bc842c44428f9679e526","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02e8bc842c44428f9679e526: start","job_status":"start","queue_duration_s":0.808403,"scheduling_latency_s":0.808425}1427vm-test-run-nixbot-gitlab> gitlab # [ 159.472265] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02e8bc842c44428f9679e526","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02e8bc842c44428f9679e526: done: 0.015533 sec","job_status":"done","queue_duration_s":0.808403,"scheduling_latency_s":0.808425,"redis_calls":13,"redis_duration_s":0.002947,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015533,"completed_at":"2026-06-17T14:54:03.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428vm-test-run-nixbot-gitlab> gitlab # [ 159.494944] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c4c3348ac87b04dddbcb22","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c4c3348ac87b04dddbcb22: start","job_status":"start","queue_duration_s":0.825409,"scheduling_latency_s":0.82543}1429vm-test-run-nixbot-gitlab> gitlab # [ 159.515051] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c4c3348ac87b04dddbcb22","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c4c3348ac87b04dddbcb22: done: 0.016378 sec","job_status":"done","queue_duration_s":0.825409,"scheduling_latency_s":0.82543,"redis_calls":13,"redis_duration_s":0.0030219999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016378,"completed_at":"2026-06-17T14:54:03.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430vm-test-run-nixbot-gitlab> gitlab # [ 159.583556] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02df8ba4fb75235ff299d8e3","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02df8ba4fb75235ff299d8e3: start","job_status":"start","queue_duration_s":0.843258,"scheduling_latency_s":0.843283}1431vm-test-run-nixbot-gitlab> gitlab # [ 159.596425] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02df8ba4fb75235ff299d8e3","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02df8ba4fb75235ff299d8e3: done: 0.015833 sec","job_status":"done","queue_duration_s":0.843258,"scheduling_latency_s":0.843283,"redis_calls":13,"redis_duration_s":0.00297,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015833,"completed_at":"2026-06-17T14:54:03.889Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432vm-test-run-nixbot-gitlab> gitlab # [ 159.660502] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b244e50fd683e89e2a272bd0","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b244e50fd683e89e2a272bd0: start","job_status":"start","queue_duration_s":0.860459,"scheduling_latency_s":0.86048}1433vm-test-run-nixbot-gitlab> gitlab # [ 159.680587] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b244e50fd683e89e2a272bd0","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b244e50fd683e89e2a272bd0: done: 0.01576 sec","job_status":"done","queue_duration_s":0.860459,"scheduling_latency_s":0.86048,"redis_calls":13,"redis_duration_s":0.003097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002155,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01576,"completed_at":"2026-06-17T14:54:03.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434vm-test-run-nixbot-gitlab> gitlab # [ 159.766314] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3aa610bc4253c65ca7d5ac12","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3aa610bc4253c65ca7d5ac12: start","job_status":"start","queue_duration_s":0.877537,"scheduling_latency_s":0.877558}1435vm-test-run-nixbot-gitlab> gitlab # [ 159.768165] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3aa610bc4253c65ca7d5ac12","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3aa610bc4253c65ca7d5ac12: done: 0.01563 sec","job_status":"done","queue_duration_s":0.877537,"scheduling_latency_s":0.877558,"redis_calls":13,"redis_duration_s":0.00311,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01563,"completed_at":"2026-06-17T14:54:03.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436vm-test-run-nixbot-gitlab> gitlab # [ 159.824185] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"874735a229829356652d9762","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-874735a229829356652d9762: start","job_status":"start","queue_duration_s":0.894459,"scheduling_latency_s":0.89448}1437vm-test-run-nixbot-gitlab> gitlab # [ 159.844064] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"874735a229829356652d9762","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-874735a229829356652d9762: done: 0.01559 sec","job_status":"done","queue_duration_s":0.894459,"scheduling_latency_s":0.89448,"redis_calls":13,"redis_duration_s":0.002954,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01559,"completed_at":"2026-06-17T14:54:03.940Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438vm-test-run-nixbot-gitlab> gitlab # [ 159.912041] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83664ac46f32999b01394705","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83664ac46f32999b01394705: start","job_status":"start","queue_duration_s":0.911326,"scheduling_latency_s":0.911346}1439vm-test-run-nixbot-gitlab> gitlab # [ 159.924282] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83664ac46f32999b01394705","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83664ac46f32999b01394705: done: 0.015824 sec","job_status":"done","queue_duration_s":0.911326,"scheduling_latency_s":0.911346,"redis_calls":13,"redis_duration_s":0.0029040000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015824,"completed_at":"2026-06-17T14:54:03.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440vm-test-run-nixbot-gitlab> gitlab # [ 159.987807] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22d7db1b9bc46dddeb5cce08","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22d7db1b9bc46dddeb5cce08: start","job_status":"start","queue_duration_s":0.928454,"scheduling_latency_s":0.928477}1441vm-test-run-nixbot-gitlab> gitlab # [ 160.007373] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22d7db1b9bc46dddeb5cce08","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22d7db1b9bc46dddeb5cce08: done: 0.015704 sec","job_status":"done","queue_duration_s":0.928454,"scheduling_latency_s":0.928477,"redis_calls":13,"redis_duration_s":0.003362,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002271,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015704,"completed_at":"2026-06-17T14:54:03.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442vm-test-run-nixbot-gitlab> gitlab # [ 160.122377] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2757cba3342bc8234404ad1d","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2757cba3342bc8234404ad1d: start","job_status":"start","queue_duration_s":0.94558,"scheduling_latency_s":0.945601}1443vm-test-run-nixbot-gitlab> gitlab # [ 160.123947] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2757cba3342bc8234404ad1d","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2757cba3342bc8234404ad1d: done: 0.015584 sec","job_status":"done","queue_duration_s":0.94558,"scheduling_latency_s":0.945601,"redis_calls":13,"redis_duration_s":0.003014,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015584,"completed_at":"2026-06-17T14:54:03.991Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444vm-test-run-nixbot-gitlab> gitlab # [ 160.150973] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:03.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8afc608468bec6db4148f1a7","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8afc608468bec6db4148f1a7: start","job_status":"start","queue_duration_s":0.962604,"scheduling_latency_s":0.962626}1445vm-test-run-nixbot-gitlab> gitlab # [ 160.170922] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8afc608468bec6db4148f1a7","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8afc608468bec6db4148f1a7: done: 0.015882 sec","job_status":"done","queue_duration_s":0.962604,"scheduling_latency_s":0.962626,"redis_calls":13,"redis_duration_s":0.0030819999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015882,"completed_at":"2026-06-17T14:54:04.008Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446vm-test-run-nixbot-gitlab> gitlab # [ 160.267946] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db9fbfcf73ec2bb15dc1d4db","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db9fbfcf73ec2bb15dc1d4db: start","job_status":"start","queue_duration_s":0.979909,"scheduling_latency_s":0.979966}1447vm-test-run-nixbot-gitlab> gitlab # [ 160.269382] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db9fbfcf73ec2bb15dc1d4db","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db9fbfcf73ec2bb15dc1d4db: done: 0.016493 sec","job_status":"done","queue_duration_s":0.979909,"scheduling_latency_s":0.979966,"redis_calls":13,"redis_duration_s":0.003972,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001407,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002565,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016493,"completed_at":"2026-06-17T14:54:04.026Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448vm-test-run-nixbot-gitlab> gitlab # [ 160.315214] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9656250a97ce9506d48a7e53","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9656250a97ce9506d48a7e53: start","job_status":"start","queue_duration_s":0.998301,"scheduling_latency_s":0.998322}1449vm-test-run-nixbot-gitlab> gitlab # [ 160.335252] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9656250a97ce9506d48a7e53","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9656250a97ce9506d48a7e53: done: 0.015773 sec","job_status":"done","queue_duration_s":0.998301,"scheduling_latency_s":0.998322,"redis_calls":13,"redis_duration_s":0.002996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015773,"completed_at":"2026-06-17T14:54:04.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450vm-test-run-nixbot-gitlab> gitlab # [ 160.412077] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23825ac1c97167d84578ee66","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23825ac1c97167d84578ee66: start","job_status":"start","queue_duration_s":1.015509,"scheduling_latency_s":1.015532}1451vm-test-run-nixbot-gitlab> gitlab # [ 160.415515] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23825ac1c97167d84578ee66","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23825ac1c97167d84578ee66: done: 0.015385 sec","job_status":"done","queue_duration_s":1.015509,"scheduling_latency_s":1.015532,"redis_calls":13,"redis_duration_s":0.002972,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015385,"completed_at":"2026-06-17T14:54:04.061Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452vm-test-run-nixbot-gitlab> gitlab # [ 160.478563] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9aad00d9fe6ee4b00e43c67","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9aad00d9fe6ee4b00e43c67: start","job_status":"start","queue_duration_s":1.03237,"scheduling_latency_s":1.032397}1453vm-test-run-nixbot-gitlab> gitlab # [ 160.556079] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9aad00d9fe6ee4b00e43c67","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9aad00d9fe6ee4b00e43c67: done: 0.015974 sec","job_status":"done","queue_duration_s":1.03237,"scheduling_latency_s":1.032397,"redis_calls":13,"redis_duration_s":0.003231,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002302,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015974,"completed_at":"2026-06-17T14:54:04.078Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454vm-test-run-nixbot-gitlab> gitlab # [ 160.559753] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbe5823f3a108ab0149ac934","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbe5823f3a108ab0149ac934: start","job_status":"start","queue_duration_s":1.049632,"scheduling_latency_s":1.049654}1455vm-test-run-nixbot-gitlab> gitlab # [ 160.639338] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbe5823f3a108ab0149ac934","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbe5823f3a108ab0149ac934: done: 0.015869 sec","job_status":"done","queue_duration_s":1.049632,"scheduling_latency_s":1.049654,"redis_calls":13,"redis_duration_s":0.003056,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015869,"completed_at":"2026-06-17T14:54:04.095Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456vm-test-run-nixbot-gitlab> gitlab # [ 160.641082] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6af55680cecb5843d7c9a10d","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6af55680cecb5843d7c9a10d: start","job_status":"start","queue_duration_s":1.066777,"scheduling_latency_s":1.066798}1457vm-test-run-nixbot-gitlab> gitlab # [ 160.660526] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6af55680cecb5843d7c9a10d","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6af55680cecb5843d7c9a10d: done: 0.015632 sec","job_status":"done","queue_duration_s":1.066777,"scheduling_latency_s":1.066798,"redis_calls":13,"redis_duration_s":0.003246,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015632,"completed_at":"2026-06-17T14:54:04.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458vm-test-run-nixbot-gitlab> gitlab # [ 160.723790] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf99442c7eb8bb28e27f895b","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf99442c7eb8bb28e27f895b: start","job_status":"start","queue_duration_s":1.083719,"scheduling_latency_s":1.08374}1459vm-test-run-nixbot-gitlab> gitlab # [ 160.743850] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf99442c7eb8bb28e27f895b","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf99442c7eb8bb28e27f895b: done: 0.015773 sec","job_status":"done","queue_duration_s":1.083719,"scheduling_latency_s":1.08374,"redis_calls":13,"redis_duration_s":0.0030689999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015773,"completed_at":"2026-06-17T14:54:04.129Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460vm-test-run-nixbot-gitlab> gitlab # [ 160.844972] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"648c1c76d81d0f0c59518952","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-648c1c76d81d0f0c59518952: start","job_status":"start","queue_duration_s":1.100829,"scheduling_latency_s":1.100851}1461vm-test-run-nixbot-gitlab> gitlab # [ 160.846445] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"648c1c76d81d0f0c59518952","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-648c1c76d81d0f0c59518952: done: 0.015419 sec","job_status":"done","queue_duration_s":1.100829,"scheduling_latency_s":1.100851,"redis_calls":13,"redis_duration_s":0.002988,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015419,"completed_at":"2026-06-17T14:54:04.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462vm-test-run-nixbot-gitlab> gitlab # [ 160.887533] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b75c06acc66932f5c6ae3ff4","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b75c06acc66932f5c6ae3ff4: start","job_status":"start","queue_duration_s":1.117676,"scheduling_latency_s":1.117697}1463vm-test-run-nixbot-gitlab> gitlab # [ 160.907412] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b75c06acc66932f5c6ae3ff4","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b75c06acc66932f5c6ae3ff4: done: 0.015413 sec","job_status":"done","queue_duration_s":1.117676,"scheduling_latency_s":1.117697,"redis_calls":13,"redis_duration_s":0.0029040000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015413,"completed_at":"2026-06-17T14:54:04.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464vm-test-run-nixbot-gitlab> gitlab # [ 160.992169] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dccf58ace32e8594484760c","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dccf58ace32e8594484760c: start","job_status":"start","queue_duration_s":1.13457,"scheduling_latency_s":1.134591}1465vm-test-run-nixbot-gitlab> gitlab # [ 160.993446] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dccf58ace32e8594484760c","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dccf58ace32e8594484760c: done: 0.015993 sec","job_status":"done","queue_duration_s":1.13457,"scheduling_latency_s":1.134591,"redis_calls":13,"redis_duration_s":0.003004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015993,"completed_at":"2026-06-17T14:54:04.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466vm-test-run-nixbot-gitlab> gitlab # [ 161.054239] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31b6f62a492d34050b3a7111","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31b6f62a492d34050b3a7111: start","job_status":"start","queue_duration_s":1.152025,"scheduling_latency_s":1.152046}1467vm-test-run-nixbot-gitlab> gitlab # [ 161.135330] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31b6f62a492d34050b3a7111","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31b6f62a492d34050b3a7111: done: 0.015506 sec","job_status":"done","queue_duration_s":1.152025,"scheduling_latency_s":1.152046,"redis_calls":13,"redis_duration_s":0.0030559999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015506,"completed_at":"2026-06-17T14:54:04.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468vm-test-run-nixbot-gitlab> gitlab # [ 161.137049] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"150261d782b61cd0fa27bdef","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-150261d782b61cd0fa27bdef: start","job_status":"start","queue_duration_s":1.169086,"scheduling_latency_s":1.169108}1469vm-test-run-nixbot-gitlab> gitlab # [ 161.154526] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"150261d782b61cd0fa27bdef","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-150261d782b61cd0fa27bdef: done: 0.015673 sec","job_status":"done","queue_duration_s":1.169086,"scheduling_latency_s":1.169108,"redis_calls":13,"redis_duration_s":0.002991,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015673,"completed_at":"2026-06-17T14:54:04.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470vm-test-run-nixbot-gitlab> gitlab # [ 161.279076] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"962bb295bb745a2249b6ec5e","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-962bb295bb745a2249b6ec5e: start","job_status":"start","queue_duration_s":1.186238,"scheduling_latency_s":1.186262}1471vm-test-run-nixbot-gitlab> gitlab # [ 161.280399] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"962bb295bb745a2249b6ec5e","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-962bb295bb745a2249b6ec5e: done: 0.015462 sec","job_status":"done","queue_duration_s":1.186238,"scheduling_latency_s":1.186262,"redis_calls":13,"redis_duration_s":0.003151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001119,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015462,"completed_at":"2026-06-17T14:54:04.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472vm-test-run-nixbot-gitlab> gitlab # [ 161.297750] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b3ba94bd391853d9ca7479f","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b3ba94bd391853d9ca7479f: start","job_status":"start","queue_duration_s":1.203199,"scheduling_latency_s":1.20322}1473vm-test-run-nixbot-gitlab> gitlab # [ 161.318287] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b3ba94bd391853d9ca7479f","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b3ba94bd391853d9ca7479f: done: 0.015581 sec","job_status":"done","queue_duration_s":1.203199,"scheduling_latency_s":1.20322,"redis_calls":13,"redis_duration_s":0.003037,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015581,"completed_at":"2026-06-17T14:54:04.249Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474vm-test-run-nixbot-gitlab> gitlab # [ 161.423527] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"664aa65b7013b30318b685c6","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-664aa65b7013b30318b685c6: start","job_status":"start","queue_duration_s":1.220185,"scheduling_latency_s":1.220206}1475vm-test-run-nixbot-gitlab> gitlab # [ 161.424862] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"664aa65b7013b30318b685c6","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-664aa65b7013b30318b685c6: done: 0.015598 sec","job_status":"done","queue_duration_s":1.220185,"scheduling_latency_s":1.220206,"redis_calls":13,"redis_duration_s":0.003152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015598,"completed_at":"2026-06-17T14:54:04.266Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476vm-test-run-nixbot-gitlab> gitlab # [ 161.461749] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc425af39b0cfef8f3d8ea42","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc425af39b0cfef8f3d8ea42: start","job_status":"start","queue_duration_s":1.237105,"scheduling_latency_s":1.237126}1477vm-test-run-nixbot-gitlab> gitlab # [ 161.482092] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc425af39b0cfef8f3d8ea42","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc425af39b0cfef8f3d8ea42: done: 0.016305 sec","job_status":"done","queue_duration_s":1.237105,"scheduling_latency_s":1.237126,"redis_calls":13,"redis_duration_s":0.0033819999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001122,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00226,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016305,"completed_at":"2026-06-17T14:54:04.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478vm-test-run-nixbot-gitlab> gitlab # [ 161.568177] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:04.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"659bf29bf9a6c0fd91cd782f","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-659bf29bf9a6c0fd91cd782f: start","job_status":"start","queue_duration_s":1.254842,"scheduling_latency_s":1.254864}1479vm-test-run-nixbot-gitlab> gitlab # [ 161.896371] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1354274396 wd_nsec: 13542735631480vm-test-run-nixbot-gitlab> gitlab # [ 161.640857] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"659bf29bf9a6c0fd91cd782f","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-659bf29bf9a6c0fd91cd782f: done: 1.918475 sec","job_status":"done","queue_duration_s":1.254842,"scheduling_latency_s":1.254864,"redis_calls":13,"redis_duration_s":0.00335,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001313,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":1.918475,"completed_at":"2026-06-17T14:54:06.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481vm-test-run-nixbot-gitlab> gitlab # [ 161.642907] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71881db08bc6cde5e9f8d6ea","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71881db08bc6cde5e9f8d6ea: start","job_status":"start","queue_duration_s":3.175059,"scheduling_latency_s":3.175089}1482vm-test-run-nixbot-gitlab> gitlab # [ 161.658882] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71881db08bc6cde5e9f8d6ea","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71881db08bc6cde5e9f8d6ea: done: 0.016455 sec","job_status":"done","queue_duration_s":3.175059,"scheduling_latency_s":3.175089,"redis_calls":13,"redis_duration_s":0.0031739999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002248,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016455,"completed_at":"2026-06-17T14:54:06.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483vm-test-run-nixbot-gitlab> gitlab # [ 161.715265] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fd0d8c2acd5367a792ea28b","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fd0d8c2acd5367a792ea28b: start","job_status":"start","queue_duration_s":3.192805,"scheduling_latency_s":3.192827}1484vm-test-run-nixbot-gitlab> gitlab # [ 161.735105] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fd0d8c2acd5367a792ea28b","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fd0d8c2acd5367a792ea28b: done: 0.015146 sec","job_status":"done","queue_duration_s":3.192805,"scheduling_latency_s":3.192827,"redis_calls":13,"redis_duration_s":0.003045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015146,"completed_at":"2026-06-17T14:54:06.238Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485vm-test-run-nixbot-gitlab> gitlab # [ 161.856294] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d072073890b06be2894db555","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d072073890b06be2894db555: start","job_status":"start","queue_duration_s":3.209328,"scheduling_latency_s":3.20935}1486vm-test-run-nixbot-gitlab> gitlab # [ 161.857915] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d072073890b06be2894db555","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d072073890b06be2894db555: done: 0.015098 sec","job_status":"done","queue_duration_s":3.209328,"scheduling_latency_s":3.20935,"redis_calls":13,"redis_duration_s":0.003112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015098,"completed_at":"2026-06-17T14:54:06.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487vm-test-run-nixbot-gitlab> gitlab # [ 161.880097] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acd020b738131a2803b0966d","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acd020b738131a2803b0966d: start","job_status":"start","queue_duration_s":3.225785,"scheduling_latency_s":3.225807}1488vm-test-run-nixbot-gitlab> gitlab # [ 161.900210] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acd020b738131a2803b0966d","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acd020b738131a2803b0966d: done: 0.015139 sec","job_status":"done","queue_duration_s":3.225785,"scheduling_latency_s":3.225807,"redis_calls":13,"redis_duration_s":0.0030919999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015139,"completed_at":"2026-06-17T14:54:06.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489vm-test-run-nixbot-gitlab> gitlab # [ 162.002403] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b360a55d90cb887fb70e2023","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b360a55d90cb887fb70e2023: start","job_status":"start","queue_duration_s":3.242383,"scheduling_latency_s":3.242414}1490vm-test-run-nixbot-gitlab> gitlab # [ 162.003990] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b360a55d90cb887fb70e2023","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b360a55d90cb887fb70e2023: done: 0.015415 sec","job_status":"done","queue_duration_s":3.242383,"scheduling_latency_s":3.242414,"redis_calls":13,"redis_duration_s":0.00293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015415,"completed_at":"2026-06-17T14:54:06.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491vm-test-run-nixbot-gitlab> gitlab # [ 162.044500] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44cd196d4ab4770016ad1c3b","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44cd196d4ab4770016ad1c3b: start","job_status":"start","queue_duration_s":3.259242,"scheduling_latency_s":3.259263}1492vm-test-run-nixbot-gitlab> gitlab # [ 162.064842] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44cd196d4ab4770016ad1c3b","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44cd196d4ab4770016ad1c3b: done: 0.015202 sec","job_status":"done","queue_duration_s":3.259242,"scheduling_latency_s":3.259263,"redis_calls":13,"redis_duration_s":0.003046,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015202,"completed_at":"2026-06-17T14:54:06.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493vm-test-run-nixbot-gitlab> gitlab # [ 162.127875] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fbdc174ebf8e5caa2e783a1","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fbdc174ebf8e5caa2e783a1: start","job_status":"start","queue_duration_s":3.275799,"scheduling_latency_s":3.275823}1494vm-test-run-nixbot-gitlab> gitlab # [ 162.147961] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fbdc174ebf8e5caa2e783a1","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fbdc174ebf8e5caa2e783a1: done: 0.015107 sec","job_status":"done","queue_duration_s":3.275799,"scheduling_latency_s":3.275823,"redis_calls":13,"redis_duration_s":0.003177,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002235,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015107,"completed_at":"2026-06-17T14:54:06.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495vm-test-run-nixbot-gitlab> gitlab # [ 162.211095] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"887f154572a24de91653d625","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-887f154572a24de91653d625: start","job_status":"start","queue_duration_s":3.292296,"scheduling_latency_s":3.292317}1496vm-test-run-nixbot-gitlab> gitlab # [ 162.231219] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"887f154572a24de91653d625","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-887f154572a24de91653d625: done: 0.015267 sec","job_status":"done","queue_duration_s":3.292296,"scheduling_latency_s":3.292317,"redis_calls":13,"redis_duration_s":0.0031450000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00101,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015267,"completed_at":"2026-06-17T14:54:06.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497vm-test-run-nixbot-gitlab> gitlab # [ 162.307426] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"881b0e487276bf2cf3ea4106","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-881b0e487276bf2cf3ea4106: start","job_status":"start","queue_duration_s":3.308863,"scheduling_latency_s":3.308884}1498vm-test-run-nixbot-gitlab> gitlab # [ 162.312318] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"881b0e487276bf2cf3ea4106","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-881b0e487276bf2cf3ea4106: done: 0.018344 sec","job_status":"done","queue_duration_s":3.308863,"scheduling_latency_s":3.308884,"redis_calls":13,"redis_duration_s":0.004675,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018344,"completed_at":"2026-06-17T14:54:06.357Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499vm-test-run-nixbot-gitlab> gitlab # [ 162.437849] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9511ddbf9431a7a27a4f923","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9511ddbf9431a7a27a4f923: start","job_status":"start","queue_duration_s":3.328917,"scheduling_latency_s":3.328973}1500vm-test-run-nixbot-gitlab> gitlab # [ 162.440221] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9511ddbf9431a7a27a4f923","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9511ddbf9431a7a27a4f923: done: 0.015459 sec","job_status":"done","queue_duration_s":3.328917,"scheduling_latency_s":3.328973,"redis_calls":13,"redis_duration_s":0.002865,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015459,"completed_at":"2026-06-17T14:54:06.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501vm-test-run-nixbot-gitlab> gitlab # [ 162.456232] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0d31334634dad0d0df5a602","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0d31334634dad0d0df5a602: start","job_status":"start","queue_duration_s":3.345821,"scheduling_latency_s":3.345841}1502vm-test-run-nixbot-gitlab> gitlab # [ 162.476254] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0d31334634dad0d0df5a602","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0d31334634dad0d0df5a602: done: 0.015637 sec","job_status":"done","queue_duration_s":3.345821,"scheduling_latency_s":3.345841,"redis_calls":13,"redis_duration_s":0.00319,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002271,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015637,"completed_at":"2026-06-17T14:54:06.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503vm-test-run-nixbot-gitlab> gitlab # [ 162.583947] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"209fefa61481636ad86665ff","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-209fefa61481636ad86665ff: start","job_status":"start","queue_duration_s":3.362879,"scheduling_latency_s":3.362903}1504vm-test-run-nixbot-gitlab> gitlab # [ 162.585421] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"209fefa61481636ad86665ff","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-209fefa61481636ad86665ff: done: 0.017044 sec","job_status":"done","queue_duration_s":3.362879,"scheduling_latency_s":3.362903,"redis_calls":13,"redis_duration_s":0.003288,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001079,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017044,"completed_at":"2026-06-17T14:54:06.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505vm-test-run-nixbot-gitlab> gitlab # [ 162.621147] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa89cb1c5fda089a8c4d57e0","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa89cb1c5fda089a8c4d57e0: start","job_status":"start","queue_duration_s":3.38131,"scheduling_latency_s":3.381331}1506vm-test-run-nixbot-gitlab> gitlab # [ 162.641175] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa89cb1c5fda089a8c4d57e0","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa89cb1c5fda089a8c4d57e0: done: 0.016061 sec","job_status":"done","queue_duration_s":3.38131,"scheduling_latency_s":3.381331,"redis_calls":13,"redis_duration_s":0.003059,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016061,"completed_at":"2026-06-17T14:54:06.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507vm-test-run-nixbot-gitlab> gitlab # [ 162.728095] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9660609d516249e1336e999f","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9660609d516249e1336e999f: start","job_status":"start","queue_duration_s":3.398729,"scheduling_latency_s":3.39875}1508vm-test-run-nixbot-gitlab> gitlab # [ 162.729827] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9660609d516249e1336e999f","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9660609d516249e1336e999f: done: 0.015634 sec","job_status":"done","queue_duration_s":3.398729,"scheduling_latency_s":3.39875,"redis_calls":13,"redis_duration_s":0.003065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015634,"completed_at":"2026-06-17T14:54:06.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509vm-test-run-nixbot-gitlab> gitlab # [ 162.785894] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0a047f1746ef151ef25899b","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0a047f1746ef151ef25899b: start","job_status":"start","queue_duration_s":3.41573,"scheduling_latency_s":3.415751}1510vm-test-run-nixbot-gitlab> gitlab # [ 162.872976] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0a047f1746ef151ef25899b","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0a047f1746ef151ef25899b: done: 0.015809 sec","job_status":"done","queue_duration_s":3.41573,"scheduling_latency_s":3.415751,"redis_calls":13,"redis_duration_s":0.003012,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015809,"completed_at":"2026-06-17T14:54:06.462Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511vm-test-run-nixbot-gitlab> gitlab # [ 162.874965] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f3eff5d5bafc83d96fc6c1c","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f3eff5d5bafc83d96fc6c1c: start","job_status":"start","queue_duration_s":3.433062,"scheduling_latency_s":3.433084}1512vm-test-run-nixbot-gitlab> gitlab # [ 162.887205] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f3eff5d5bafc83d96fc6c1c","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f3eff5d5bafc83d96fc6c1c: done: 0.015813 sec","job_status":"done","queue_duration_s":3.433062,"scheduling_latency_s":3.433084,"redis_calls":13,"redis_duration_s":0.003004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015813,"completed_at":"2026-06-17T14:54:06.479Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513vm-test-run-nixbot-gitlab> gitlab # [ 163.007784] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f8615f8a746ec18cf499bc6","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f8615f8a746ec18cf499bc6: start","job_status":"start","queue_duration_s":3.450239,"scheduling_latency_s":3.450262}1514vm-test-run-nixbot-gitlab> gitlab # [ 163.009264] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f8615f8a746ec18cf499bc6","created_at":"2026-06-17T14:54:02.899Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-dff20bf56c104c2a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"06720319d25b4fac65b98e8df3d8a1da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.030Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f8615f8a746ec18cf499bc6: done: 0.015853 sec","job_status":"done","queue_duration_s":3.450239,"scheduling_latency_s":3.450262,"redis_calls":13,"redis_duration_s":0.003321,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015853,"completed_at":"2026-06-17T14:54:06.496Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515vm-test-run-nixbot-gitlab> gitlab # [ 163.032285] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"d073d6bab889d1b0bbf5902e","created_at":"2026-06-17T14:54:02.790Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54ce886163be6675628c0196724769ed","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.054Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-d073d6bab889d1b0bbf5902e: start","job_status":"start","queue_duration_s":3.444363,"scheduling_latency_s":3.444406}1516vm-test-run-nixbot-gitlab> gitlab # [ 163.055925] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.520Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"d073d6bab889d1b0bbf5902e","created_at":"2026-06-17T14:54:02.790Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54ce886163be6675628c0196724769ed","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.054Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-d073d6bab889d1b0bbf5902e: done: 0.021391 sec","job_status":"done","queue_duration_s":3.444363,"scheduling_latency_s":3.444406,"redis_calls":12,"redis_duration_s":0.0030210000000000002,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001369,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000373511,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.021391,"completed_at":"2026-06-17T14:54:06.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00066,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517vm-test-run-nixbot-gitlab> gitlab # [ 163.164370] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1aa3976f7a7f63befcb2e478","created_at":"2026-06-17T14:54:03.060Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"656c737f5458c3abda539626040efa02","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.068Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1aa3976f7a7f63befcb2e478: start","job_status":"start","queue_duration_s":3.45306,"scheduling_latency_s":3.453081}1518vm-test-run-nixbot-gitlab> gitlab # [ 163.166089] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.542Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1aa3976f7a7f63befcb2e478","created_at":"2026-06-17T14:54:03.060Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"656c737f5458c3abda539626040efa02","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.068Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1aa3976f7a7f63befcb2e478: done: 0.020801 sec","job_status":"done","queue_duration_s":3.45306,"scheduling_latency_s":3.453081,"redis_calls":12,"redis_duration_s":0.00285,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000473,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000494756,"exclusive_lock_hold_duration_s":0.003982628,"duration_s":0.020801,"completed_at":"2026-06-17T14:54:06.542Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519vm-test-run-nixbot-gitlab> gitlab # [ 163.219205] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.543Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"22a3a4412d5613e852e486ad","created_at":"2026-06-17T14:54:03.074Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8583fb381e915168db9f216c3f2a8f42","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.083Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-22a3a4412d5613e852e486ad: start","job_status":"start","queue_duration_s":3.46009,"scheduling_latency_s":3.460111}1520vm-test-run-nixbot-gitlab> gitlab # [ 163.242162] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.876Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"22a3a4412d5613e852e486ad","created_at":"2026-06-17T14:54:03.074Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8583fb381e915168db9f216c3f2a8f42","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.083Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-22a3a4412d5613e852e486ad: done: 0.332461 sec","job_status":"done","queue_duration_s":3.46009,"scheduling_latency_s":3.460111,"redis_calls":14,"redis_duration_s":0.003613,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000996,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001398,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001219,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.285364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000446426,"exclusive_lock_hold_duration_s":0.311100153,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.332461,"completed_at":"2026-06-17T14:54:06.876Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046871,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521vm-test-run-nixbot-gitlab> gitlab # [ 163.313847] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.877Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"3ea8a5db0a86dc6bd92fc46b","created_at":"2026-06-17T14:54:03.086Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.151Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-3ea8a5db0a86dc6bd92fc46b: start","job_status":"start","queue_duration_s":3.726661,"scheduling_latency_s":3.726683}1522vm-test-run-nixbot-gitlab> gitlab # [ 163.335323] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.907Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"3ea8a5db0a86dc6bd92fc46b","created_at":"2026-06-17T14:54:03.086Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.151Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-3ea8a5db0a86dc6bd92fc46b: done: 0.029807 sec","job_status":"done","queue_duration_s":3.726661,"scheduling_latency_s":3.726683,"redis_calls":19,"redis_duration_s":0.004394,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000923,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000423,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002292,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000756,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000357588,"duration_s":0.029807,"completed_at":"2026-06-17T14:54:06.907Z","load_balancing_strategy":"primary","db_duration_s":0.001525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523vm-test-run-nixbot-gitlab> gitlab # [ 163.404184] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.909Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"95c6ad906917a4b5dff691d7","created_at":"2026-06-17T14:54:03.179Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37a0057be1629542a4bfecf0bcbcba2b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.193Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-95c6ad906917a4b5dff691d7: start","job_status":"start","queue_duration_s":3.715601,"scheduling_latency_s":3.715622}1524vm-test-run-nixbot-gitlab> gitlab # [ 163.427356] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.920Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1525vm-test-run-nixbot-gitlab> gitlab # [ 163.432894] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"95c6ad906917a4b5dff691d7","created_at":"2026-06-17T14:54:03.179Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37a0057be1629542a4bfecf0bcbcba2b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.193Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-95c6ad906917a4b5dff691d7: done: 0.016817 sec","job_status":"done","queue_duration_s":3.715601,"scheduling_latency_s":3.715622,"redis_calls":10,"redis_duration_s":0.002253,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384685,"duration_s":0.016817,"completed_at":"2026-06-17T14:54:06.926Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526vm-test-run-nixbot-gitlab> gitlab # [ 163.501181] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.927Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"7bb7f7146d75a934f55bb13c","created_at":"2026-06-17T14:54:03.275Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccc9a29b30b1e3f350fce683b75c31af","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.302Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-7bb7f7146d75a934f55bb13c: start","job_status":"start","queue_duration_s":3.625259,"scheduling_latency_s":3.625279}1527vm-test-run-nixbot-gitlab> gitlab # [ 163.523444] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"7bb7f7146d75a934f55bb13c","created_at":"2026-06-17T14:54:03.275Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccc9a29b30b1e3f350fce683b75c31af","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.302Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-7bb7f7146d75a934f55bb13c: done: 0.027187 sec","job_status":"done","queue_duration_s":3.625259,"scheduling_latency_s":3.625279,"redis_calls":10,"redis_duration_s":0.00243,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000462,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001462,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000351721,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.027187,"completed_at":"2026-06-17T14:54:06.954Z","load_balancing_strategy":"primary","db_duration_s":0.008136,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528vm-test-run-nixbot-gitlab> gitlab # [ 163.592263] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"40576d4a9ae719340fb32fcb","created_at":"2026-06-17T14:54:03.313Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db0be85ec14b86e2dfdd76333c76c9f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-17T14:54:03.327Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-40576d4a9ae719340fb32fcb: start","job_status":"start","queue_duration_s":3.628364,"scheduling_latency_s":3.628384}1529vm-test-run-nixbot-gitlab> gitlab # [ 163.613213] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.982Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"40576d4a9ae719340fb32fcb","created_at":"2026-06-17T14:54:03.313Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db0be85ec14b86e2dfdd76333c76c9f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-17T14:54:03.327Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-40576d4a9ae719340fb32fcb: done: 0.026285 sec","job_status":"done","queue_duration_s":3.628364,"scheduling_latency_s":3.628384,"redis_calls":10,"redis_duration_s":0.002401,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001457,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000388596,"duration_s":0.026285,"completed_at":"2026-06-17T14:54:06.982Z","load_balancing_strategy":"primary","db_duration_s":0.002777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530vm-test-run-nixbot-gitlab> gitlab # [ 163.679865] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:06.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5cb6cd27f422d575594d5331","created_at":"2026-06-17T14:54:03.333Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f45c0b9ebbe21640c547974255f20e8b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.345Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5cb6cd27f422d575594d5331: start","job_status":"start","queue_duration_s":3.638532,"scheduling_latency_s":3.638553}1531vm-test-run-nixbot-gitlab> gitlab # [ 163.703364] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.041Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5cb6cd27f422d575594d5331","created_at":"2026-06-17T14:54:03.333Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f45c0b9ebbe21640c547974255f20e8b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.345Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5cb6cd27f422d575594d5331: done: 0.057094 sec","job_status":"done","queue_duration_s":3.638532,"scheduling_latency_s":3.638553,"redis_calls":12,"redis_duration_s":0.005758999999999999,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000811,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002534,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001402972,"exclusive_lock_hold_duration_s":0.018790375999999998,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.057094,"completed_at":"2026-06-17T14:54:07.041Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532vm-test-run-nixbot-gitlab> gitlab # [ 163.775501] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"acefb500429b8cdae1f14e27","created_at":"2026-06-17T14:54:03.422Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67bc9fd292b5c3ad1ec310c075be307c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.440Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-acefb500429b8cdae1f14e27: start","job_status":"start","queue_duration_s":3.602892,"scheduling_latency_s":3.602922}1533vm-test-run-nixbot-gitlab> gitlab # [ 163.799291] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"acefb500429b8cdae1f14e27","created_at":"2026-06-17T14:54:03.422Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67bc9fd292b5c3ad1ec310c075be307c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.440Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-acefb500429b8cdae1f14e27: done: 0.020364 sec","job_status":"done","queue_duration_s":3.602892,"scheduling_latency_s":3.602922,"redis_calls":12,"redis_duration_s":0.002858,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000917,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000385804,"duration_s":0.020364,"completed_at":"2026-06-17T14:54:07.063Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000658,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534vm-test-run-nixbot-gitlab> gitlab # [ 163.897395] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"668b63a435c1ef7adc94297a","created_at":"2026-06-17T14:54:03.468Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a377a9635b65dbde807d4820f96daf77","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.479Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-668b63a435c1ef7adc94297a: start","job_status":"start","queue_duration_s":3.584982,"scheduling_latency_s":3.585003}1535vm-test-run-nixbot-gitlab> gitlab # [ 163.899075] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.105Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"668b63a435c1ef7adc94297a","created_at":"2026-06-17T14:54:03.468Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a377a9635b65dbde807d4820f96daf77","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.479Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-668b63a435c1ef7adc94297a: done: 0.041318 sec","job_status":"done","queue_duration_s":3.584982,"scheduling_latency_s":3.585003,"redis_calls":14,"redis_duration_s":0.003537,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000866,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001431,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00124,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000594768,"exclusive_lock_hold_duration_s":0.021365285,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.041318,"completed_at":"2026-06-17T14:54:07.105Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536vm-test-run-nixbot-gitlab> gitlab # [ 163.961180] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.107Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"43e42e9c736e7cb60c71f27c","created_at":"2026-06-17T14:54:03.509Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb211e74605f398205114b7f50b77c59","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.524Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-43e42e9c736e7cb60c71f27c: start","job_status":"start","queue_duration_s":3.582445,"scheduling_latency_s":3.582467}1537vm-test-run-nixbot-gitlab> gitlab # [ 164.041876] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.137Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"43e42e9c736e7cb60c71f27c","created_at":"2026-06-17T14:54:03.509Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb211e74605f398205114b7f50b77c59","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.524Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-43e42e9c736e7cb60c71f27c: done: 0.030523 sec","job_status":"done","queue_duration_s":3.582445,"scheduling_latency_s":3.582467,"redis_calls":13,"redis_duration_s":0.0048,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003431,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008779887,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.030523,"completed_at":"2026-06-17T14:54:07.137Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002689,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1538vm-test-run-nixbot-gitlab> gitlab # [ 164.053623] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.139Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cc25b04dccf40197518da265","created_at":"2026-06-17T14:54:03.533Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98b41bb31e78deeaa8d51e9205fb58a6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.537Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cc25b04dccf40197518da265: start","job_status":"start","queue_duration_s":3.601534,"scheduling_latency_s":3.601557}1539vm-test-run-nixbot-gitlab> gitlab # [ 164.075259] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.162Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cc25b04dccf40197518da265","created_at":"2026-06-17T14:54:03.533Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98b41bb31e78deeaa8d51e9205fb58a6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.537Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cc25b04dccf40197518da265: done: 0.023471 sec","job_status":"done","queue_duration_s":3.601534,"scheduling_latency_s":3.601557,"redis_calls":12,"redis_duration_s":0.0039700000000000004,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002102,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00132,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00050621,"exclusive_lock_hold_duration_s":0.004521525,"duration_s":0.023471,"completed_at":"2026-06-17T14:54:07.162Z","load_balancing_strategy":"primary","db_duration_s":0.00207,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540vm-test-run-nixbot-gitlab> gitlab # [ 164.185445] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"527a70aa13116c7db0ae12db","created_at":"2026-06-17T14:54:03.543Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d00e97a9e22f0206b9ea591cbdc1094","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.547Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-527a70aa13116c7db0ae12db: start","job_status":"start","queue_duration_s":3.616649,"scheduling_latency_s":3.61667}1541vm-test-run-nixbot-gitlab> gitlab # [ 164.186791] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"527a70aa13116c7db0ae12db","created_at":"2026-06-17T14:54:03.543Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d00e97a9e22f0206b9ea591cbdc1094","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.547Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-527a70aa13116c7db0ae12db: done: 0.021024 sec","job_status":"done","queue_duration_s":3.616649,"scheduling_latency_s":3.61667,"redis_calls":10,"redis_duration_s":0.00231,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386362,"duration_s":0.021024,"completed_at":"2026-06-17T14:54:07.185Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002112,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542vm-test-run-nixbot-gitlab> gitlab # [ 164.232150] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.186Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"29a3a2ada84c81a473a4201d","created_at":"2026-06-17T14:54:03.556Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"266a2347e7256476f9991f2ba48bd3a3","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.560Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-29a3a2ada84c81a473a4201d: start","job_status":"start","queue_duration_s":3.62658,"scheduling_latency_s":3.6266}1543vm-test-run-nixbot-gitlab> gitlab # [ 164.254958] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.214Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"29a3a2ada84c81a473a4201d","created_at":"2026-06-17T14:54:03.556Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"266a2347e7256476f9991f2ba48bd3a3","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.560Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-29a3a2ada84c81a473a4201d: done: 0.027468 sec","job_status":"done","queue_duration_s":3.62658,"scheduling_latency_s":3.6266,"redis_calls":16,"redis_duration_s":0.004211,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002037,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000507,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000548673,"duration_s":0.027468,"completed_at":"2026-06-17T14:54:07.214Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003507,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544vm-test-run-nixbot-gitlab> gitlab # [ 164.330380] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.215Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"22f871a59aa981af13c9be8f","created_at":"2026-06-17T14:54:03.568Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4a31f9daca288f48058a8caff24bb7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.572Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-22f871a59aa981af13c9be8f: start","job_status":"start","queue_duration_s":3.643313,"scheduling_latency_s":3.643333}1545vm-test-run-nixbot-gitlab> gitlab # [ 164.343399] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"22f871a59aa981af13c9be8f","created_at":"2026-06-17T14:54:03.568Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4a31f9daca288f48058a8caff24bb7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.572Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-22f871a59aa981af13c9be8f: done: 0.017586 sec","job_status":"done","queue_duration_s":3.643313,"scheduling_latency_s":3.643333,"redis_calls":10,"redis_duration_s":0.002627,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001647,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037826,"duration_s":0.017586,"completed_at":"2026-06-17T14:54:07.233Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546vm-test-run-nixbot-gitlab> gitlab # [ 164.474351] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.234Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"50e31dfb916f1d13a6dc4330","created_at":"2026-06-17T14:54:03.580Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"315b35ec0630d26079779dd98635e973","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.584Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-50e31dfb916f1d13a6dc4330: start","job_status":"start","queue_duration_s":3.650237,"scheduling_latency_s":3.650258}1547vm-test-run-nixbot-gitlab> gitlab # [ 164.475982] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.252Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"50e31dfb916f1d13a6dc4330","created_at":"2026-06-17T14:54:03.580Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"315b35ec0630d26079779dd98635e973","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.584Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-50e31dfb916f1d13a6dc4330: done: 0.01789 sec","job_status":"done","queue_duration_s":3.650237,"scheduling_latency_s":3.650258,"redis_calls":12,"redis_duration_s":0.003022,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000522,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000526,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000645,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345575,"duration_s":0.01789,"completed_at":"2026-06-17T14:54:07.252Z","load_balancing_strategy":"primary","db_duration_s":0.000385,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548vm-test-run-nixbot-gitlab> gitlab # [ 164.498879] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8bb9ccc880cb649375a6da6d","created_at":"2026-06-17T14:54:03.587Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d313ca73754f5a16513eaace726b695","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.617Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8bb9ccc880cb649375a6da6d: start","job_status":"start","queue_duration_s":3.636604,"scheduling_latency_s":3.636625}1549vm-test-run-nixbot-gitlab> gitlab # [ 164.522352] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.264Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1550vm-test-run-nixbot-gitlab> gitlab # [ 164.528418] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.271Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8bb9ccc880cb649375a6da6d","created_at":"2026-06-17T14:54:03.587Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d313ca73754f5a16513eaace726b695","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.617Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8bb9ccc880cb649375a6da6d: done: 0.017155 sec","job_status":"done","queue_duration_s":3.636604,"scheduling_latency_s":3.636625,"redis_calls":10,"redis_duration_s":0.002205,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000433,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001327,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000337473,"duration_s":0.017155,"completed_at":"2026-06-17T14:54:07.271Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551vm-test-run-nixbot-gitlab> gitlab # [ 164.619198] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.272Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"4e89983f33a0167aae6d7236","created_at":"2026-06-17T14:54:03.630Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"149ac774ae97a0a7f4359c7e4fd93956","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.639Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-4e89983f33a0167aae6d7236: start","job_status":"start","queue_duration_s":3.633124,"scheduling_latency_s":3.633145}1552vm-test-run-nixbot-gitlab> gitlab # [ 164.620571] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.296Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"4e89983f33a0167aae6d7236","created_at":"2026-06-17T14:54:03.630Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"149ac774ae97a0a7f4359c7e4fd93956","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.639Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-4e89983f33a0167aae6d7236: done: 0.023797 sec","job_status":"done","queue_duration_s":3.633124,"scheduling_latency_s":3.633145,"redis_calls":10,"redis_duration_s":0.0055309999999999995,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00045,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004617,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037826,"duration_s":0.023797,"completed_at":"2026-06-17T14:54:07.296Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002126,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553vm-test-run-nixbot-gitlab> gitlab # [ 164.684400] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ce2fa64a60add95b41002e54","created_at":"2026-06-17T14:54:03.769Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56e4b97807a7cd025ea65e77ae676c3f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.787Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ce2fa64a60add95b41002e54: start","job_status":"start","queue_duration_s":3.510362,"scheduling_latency_s":3.510383}1554vm-test-run-nixbot-gitlab> gitlab # [ 164.762638] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ce2fa64a60add95b41002e54","created_at":"2026-06-17T14:54:03.769Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56e4b97807a7cd025ea65e77ae676c3f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:03.787Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ce2fa64a60add95b41002e54: done: 0.413815 sec","job_status":"done","queue_duration_s":3.510362,"scheduling_latency_s":3.510383,"redis_calls":574,"redis_duration_s":0.128393,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.126507,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.316368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000368203,"duration_s":0.413815,"completed_at":"2026-06-17T14:54:07.711Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555vm-test-run-nixbot-gitlab> gitlab # [ 164.772435] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.713Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b25aadb161bb1ecd7cb83959","created_at":"2026-06-17T14:54:06.892Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-a21232997cd5458a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:06.901Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b25aadb161bb1ecd7cb83959: start","job_status":"start","queue_duration_s":0.812042,"scheduling_latency_s":0.812065}1556vm-test-run-nixbot-gitlab> gitlab # [ 164.792457] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.729Z","message":"No existing merge request to be cleaned up."}1557vm-test-run-nixbot-gitlab> gitlab # [ 164.795332] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.740Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b25aadb161bb1ecd7cb83959","created_at":"2026-06-17T14:54:06.892Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-a21232997cd5458a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:06.901Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b25aadb161bb1ecd7cb83959: done: 0.027176 sec","job_status":"done","queue_duration_s":0.812042,"scheduling_latency_s":0.812065,"redis_calls":13,"redis_duration_s":0.003781,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386362,"duration_s":0.027176,"completed_at":"2026-06-17T14:54:07.740Z","load_balancing_strategy":"primary","db_duration_s":0.004531,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558vm-test-run-nixbot-gitlab> gitlab # [ 164.907235] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.742Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"967a29a0648bad2708a7cfd9","created_at":"2026-06-17T14:54:06.892Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-a21232997cd5458a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:06.901Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-967a29a0648bad2708a7cfd9: start","job_status":"start","queue_duration_s":0.840667,"scheduling_latency_s":0.840688}1559vm-test-run-nixbot-gitlab> gitlab # [ 164.908409] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.752Z","message":"No existing merge request to be cleaned up."}1560vm-test-run-nixbot-gitlab> gitlab # [ 164.909456] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.758Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"967a29a0648bad2708a7cfd9","created_at":"2026-06-17T14:54:06.892Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-a21232997cd5458a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:06.901Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-967a29a0648bad2708a7cfd9: done: 0.016278 sec","job_status":"done","queue_duration_s":0.840667,"scheduling_latency_s":0.840688,"redis_calls":10,"redis_duration_s":0.002667,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001721,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016278,"completed_at":"2026-06-17T14:54:07.758Z","load_balancing_strategy":"primary","db_duration_s":0.001513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561vm-test-run-nixbot-gitlab> gitlab # [ 164.942187] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.759Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"abd4e5bf368acfa8b1de97d7","created_at":"2026-06-17T14:54:06.892Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-a21232997cd5458a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:06.901Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-abd4e5bf368acfa8b1de97d7: start","job_status":"start","queue_duration_s":0.858611,"scheduling_latency_s":0.858632}1562vm-test-run-nixbot-gitlab> gitlab # [ 164.962221] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.769Z","message":"No existing merge request to be cleaned up."}1563vm-test-run-nixbot-gitlab> gitlab # [ 164.965101] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.775Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"abd4e5bf368acfa8b1de97d7","created_at":"2026-06-17T14:54:06.892Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-a21232997cd5458a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:06.901Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-abd4e5bf368acfa8b1de97d7: done: 0.015787 sec","job_status":"done","queue_duration_s":0.858611,"scheduling_latency_s":0.858632,"redis_calls":10,"redis_duration_s":0.00238,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001438,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015787,"completed_at":"2026-06-17T14:54:07.775Z","load_balancing_strategy":"primary","db_duration_s":0.001417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564vm-test-run-nixbot-gitlab> gitlab # [ 165.052738] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.777Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2e757005608163cc32b70b28","created_at":"2026-06-17T14:54:06.892Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-a21232997cd5458a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:06.901Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2e757005608163cc32b70b28: start","job_status":"start","queue_duration_s":0.875836,"scheduling_latency_s":0.875857}1565vm-test-run-nixbot-gitlab> gitlab # [ 165.054236] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.787Z","message":"No existing merge request to be cleaned up."}1566vm-test-run-nixbot-gitlab> gitlab # [ 165.055533] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:07.793Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2e757005608163cc32b70b28","created_at":"2026-06-17T14:54:06.892Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-a21232997cd5458a","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9b895b7bb5f15b5fc29230532ae545b3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:06.901Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2e757005608163cc32b70b28: done: 0.015934 sec","job_status":"done","queue_duration_s":0.875836,"scheduling_latency_s":0.875857,"redis_calls":10,"redis_duration_s":0.002581,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001585,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015934,"completed_at":"2026-06-17T14:54:07.793Z","load_balancing_strategy":"primary","db_duration_s":0.001362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567vm-test-run-nixbot-gitlab> gitlab # [ 176.548440] setup-gitlab-start[2767]: W, [2026-06-17T14:54:21.110667 #2767] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1568vm-test-run-nixbot-gitlab> gitlab # [ 199.595606] runuser[2766]: pam_unix(runuser:session): session closed for user gitlab1569vm-test-run-nixbot-gitlab> gitlab # [ 199.600195] setup-gitlab-start[2991]: +++ date -d '+7 days' +%F1570vm-test-run-nixbot-gitlab> gitlab # [ 199.604274] setup-gitlab-start[2989]: ++ curl -fs -X POST -H 'Authorization: Bearer 69f526118babb9072f36fc6d073f255f60d20abf9dd87392e1efb81a4e7972b6' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-06-24"}' http://gitlab/api/v4/users/1/personal_access_tokens1571vm-test-run-nixbot-gitlab> gitlab # [ 199.607628] setup-gitlab-start[2990]: ++ jq -r .token1572vm-test-run-nixbot-gitlab> gitlab # [ 199.964759] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:44.526Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f27610d6dc04905766af11c3","created_at":"2026-06-17T14:54:44.443Z","trace_propagation_headers":{"sentry-trace":"669503caec8549e999b1c6c8cfaf272b-61ef91da62ed423d","baggage":"sentry-trace_id=669503caec8549e999b1c6c8cfaf272b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17MYH4QZEM494QTGEK8XQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9c434829509957b6e55b619bbe14c1930e4cbadc89fd07f4f905130b68c42ce1","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:44.513Z","job_size_bytes":451,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f27610d6dc04905766af11c3: start","job_status":"start","queue_duration_s":0.012483,"scheduling_latency_s":0.012557}1573vm-test-run-nixbot-gitlab> gitlab # [ 200.012896] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:44 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 3981574vm-test-run-nixbot-gitlab> gitlab # [ 200.017764] setup-gitlab-start[2717]: + TOKEN=glpat-sHOx8I_Ie75OwkI96H42_G86MQp1OjEH.01.0w1ma9rtm1575vm-test-run-nixbot-gitlab> gitlab # [ 200.019923] setup-gitlab-start[2717]: + test -n glpat-sHOx8I_Ie75OwkI96H42_G86MQp1OjEH.01.0w1ma9rtm1576vm-test-run-nixbot-gitlab> gitlab # [ 200.022110] setup-gitlab-start[2717]: + test glpat-sHOx8I_Ie75OwkI96H42_G86MQp1OjEH.01.0w1ma9rtm '!=' null1577vm-test-run-nixbot-gitlab> gitlab # [ 200.024303] setup-gitlab-start[2717]: + mkdir -p /var/lib/secrets1578vm-test-run-nixbot-gitlab> gitlab # [ 200.026466] setup-gitlab-start[2717]: + echo glpat-sHOx8I_Ie75OwkI96H42_G86MQp1OjEH.01.0w1ma9rtm1579vm-test-run-nixbot-gitlab> gitlab # [ 200.028146] setup-gitlab-start[2717]: + chmod 644 /var/lib/secrets/gitlab-token1580vm-test-run-nixbot-gitlab> gitlab # [ 200.029629] setup-gitlab-start[2717]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-sHOx8I_Ie75OwkI96H42_G86MQp1OjEH.01.0w1ma9rtm' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1581vm-test-run-nixbot-gitlab> gitlab # [ 200.170883] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-06-17T14:54:44.732Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f27610d6dc04905766af11c3","created_at":"2026-06-17T14:54:44.443Z","trace_propagation_headers":{"sentry-trace":"669503caec8549e999b1c6c8cfaf272b-61ef91da62ed423d","baggage":"sentry-trace_id=669503caec8549e999b1c6c8cfaf272b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17MYH4QZEM494QTGEK8XQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9c434829509957b6e55b619bbe14c1930e4cbadc89fd07f4f905130b68c42ce1","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:44.513Z","job_size_bytes":451,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f27610d6dc04905766af11c3: fail: 0.205571 sec","job_status":"fail","queue_duration_s":0.012483,"scheduling_latency_s":0.012557,"redis_calls":11,"redis_duration_s":0.006866,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001804,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003574,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001128355,"duration_s":0.205571,"completed_at":"2026-06-17T14:54:44.731Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.050562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582vm-test-run-nixbot-gitlab> gitlab # [ 201.565910] gitaly[1386]: time="2026-06-17T14:54:46.128Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.127" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583vm-test-run-nixbot-gitlab> gitlab # [ 201.570106] gitaly[1386]: time="2026-06-17T14:54:46.128Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.127" grpc.time_ms=0.698 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1584vm-test-run-nixbot-gitlab> gitlab # [ 201.755052] gitaly[1386]: time="2026-06-17T14:54:46.318Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.196" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.317" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1585vm-test-run-nixbot-gitlab> gitlab # [ 201.757175] gitaly[1386]: time="2026-06-17T14:54:46.318Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.196" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.317" grpc.time_ms=0.517 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1586vm-test-run-nixbot-gitlab> gitlab # [ 201.803827] gitaly[1386]: time="2026-06-17T14:54:46.367Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G diskcache=f9b29d38-d646-44d3-ae03-45697dd71e9e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-17T14:54:56.212" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.333" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1587vm-test-run-nixbot-gitlab> gitlab # [ 201.806665] gitaly[1386]: time="2026-06-17T14:54:46.367Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=32 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-17T14:54:56.212" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.333" grpc.time_ms=33.925 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1588vm-test-run-nixbot-gitlab> gitlab # [ 201.824369] gitaly[1386]: time="2026-06-17T14:54:46.387Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.265" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.386" pid=1386 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1589vm-test-run-nixbot-gitlab> gitlab # [ 201.825855] gitaly[1386]: time="2026-06-17T14:54:46.387Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.265" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.386" grpc.time_ms=1.002 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1590vm-test-run-nixbot-gitlab> gitlab # [ 201.844560] gitaly[1386]: time="2026-06-17T14:54:46.407Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-17T14:54:56.274" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.395" grpc.time_ms=11.909 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1591vm-test-run-nixbot-gitlab> gitlab # [ 201.876822] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:46.438Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e28071b88db995bab8c1a20c","created_at":"2026-06-17T14:54:46.420Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.435Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e28071b88db995bab8c1a20c: start","job_status":"start","queue_duration_s":0.002585,"scheduling_latency_s":0.002648}1592vm-test-run-nixbot-gitlab> gitlab # [ 201.894770] gitaly[1386]: time="2026-06-17T14:54:46.457Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.457" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1593vm-test-run-nixbot-gitlab> gitlab # [ 201.896265] gitaly[1386]: time="2026-06-17T14:54:46.458Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.457" grpc.time_ms=0.54 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1594vm-test-run-nixbot-gitlab> gitlab # [ 201.932227] gitaly[1386]: time="2026-06-17T14:54:46.495Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G diskcache=585eb5ae-d166-42af-8eba-ce1bff4eabdc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-17T14:54:56.010" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.461" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1595vm-test-run-nixbot-gitlab> gitlab # [ 201.933713] gitaly[1386]: time="2026-06-17T14:54:46.495Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=32 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-17T14:54:56.010" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.461" grpc.time_ms=34.206 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1596vm-test-run-nixbot-gitlab> gitlab # [ 201.949614] gitaly[1386]: time="2026-06-17T14:54:46.502Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.052" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.502" pid=1386 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1597vm-test-run-nixbot-gitlab> gitlab # [ 201.966733] gitaly[1386]: time="2026-06-17T14:54:46.502Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.052" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.502" grpc.time_ms=0.461 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1598vm-test-run-nixbot-gitlab> gitlab # [ 201.984817] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:46.503Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e28071b88db995bab8c1a20c","created_at":"2026-06-17T14:54:46.420Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.435Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e28071b88db995bab8c1a20c: done: 0.065365 sec","job_status":"done","queue_duration_s":0.002585,"scheduling_latency_s":0.002648,"redis_calls":10,"redis_duration_s":0.003562,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001931,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001631,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.042682519,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.065365,"completed_at":"2026-06-17T14:54:46.503Z","load_balancing_strategy":"primary","db_duration_s":0.014775,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1599vm-test-run-nixbot-gitlab> gitlab # [ 202.054974] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:46.505Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f1e60c903a30cb076b420338","created_at":"2026-06-17T14:54:46.442Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.450Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f1e60c903a30cb076b420338: start","job_status":"start","queue_duration_s":0.05426,"scheduling_latency_s":0.054284}1600vm-test-run-nixbot-gitlab> gitlab # [ 202.078445] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:46.579Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f1e60c903a30cb076b420338","created_at":"2026-06-17T14:54:46.442Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.450Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f1e60c903a30cb076b420338: done: 0.074822 sec","job_status":"done","queue_duration_s":0.05426,"scheduling_latency_s":0.054284,"redis_calls":14,"redis_duration_s":0.003574,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000793,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001652,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001129,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.054652477000000005,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.074822,"completed_at":"2026-06-17T14:54:46.579Z","load_balancing_strategy":"primary","db_duration_s":0.016387,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1601vm-test-run-nixbot-gitlab> gitlab # [ 202.157315] gitaly[1386]: time="2026-06-17T14:54:46.659Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=151 command.spawn_token_fork_ms=146 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-17T14:54:56.055" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.506" grpc.time_ms=152.763 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1602vm-test-run-nixbot-gitlab> gitlab # [ 202.171301] gitaly[1386]: time="2026-06-17T14:54:46.680Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:56.230" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.680" grpc.time_ms=0.2 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603vm-test-run-nixbot-gitlab> gitlab # [ 202.188946] gitaly[1386]: time="2026-06-17T14:54:46.686Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-17T14:54:56.236" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.686" grpc.time_ms=0.174 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1604vm-test-run-nixbot-gitlab> gitlab # [ 202.267710] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:46.830Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"2464de660e6486f623ef5f33","created_at":"2026-06-17T14:54:46.823Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.828Z","job_size_bytes":5,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-2464de660e6486f623ef5f33: start","job_status":"start","queue_duration_s":0.00167,"scheduling_latency_s":0.001713}1605vm-test-run-nixbot-gitlab> gitlab # [ 202.321128] gitaly[1386]: time="2026-06-17T14:54:46.884Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17NC81JAFW62YC3ZGZ69G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-17T14:54:56.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:46.876" grpc.time_ms=6.604 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1606vm-test-run-nixbot-gitlab> gitlab # [ 202.360058] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:46.922Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"2464de660e6486f623ef5f33","created_at":"2026-06-17T14:54:46.823Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.828Z","job_size_bytes":5,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-2464de660e6486f623ef5f33: done: 0.092365 sec","job_status":"done","queue_duration_s":0.00167,"scheduling_latency_s":0.001713,"redis_calls":12,"redis_duration_s":0.00367,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001134,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002078,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000358426,"duration_s":0.092365,"completed_at":"2026-06-17T14:54:46.922Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013352,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1607vm-test-run-nixbot-gitlab> gitlab # [ 202.362199] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:46.924Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"5011425078d10df9274fb56b","created_at":"2026-06-17T14:54:46.853Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.858Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-5011425078d10df9274fb56b: start","job_status":"start","queue_duration_s":0.066457,"scheduling_latency_s":0.06648}1608vm-test-run-nixbot-gitlab> gitlab # [ 202.662124] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:47 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 26071609vm-test-run-nixbot-gitlab> gitlab # [ 202.671306] setup-gitlab-start[2717]: + mkdir -p /tmp/test-flake1610vm-test-run-nixbot-gitlab> gitlab # [ 202.675228] setup-gitlab-start[2717]: + cd /tmp/test-flake1611vm-test-run-nixbot-gitlab> gitlab # [ 202.677774] setup-gitlab-start[2717]: + git init -b master1612vm-test-run-nixbot-gitlab> gitlab # [ 202.684798] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:47.247Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"5011425078d10df9274fb56b","created_at":"2026-06-17T14:54:46.853Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.858Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-5011425078d10df9274fb56b: done: 0.323194 sec","job_status":"done","queue_duration_s":0.066457,"scheduling_latency_s":0.06648,"redis_calls":11,"redis_duration_s":0.008504,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001045,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006863,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000596,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.121,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.188,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.188,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.12,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000381054,"duration_s":0.323194,"completed_at":"2026-06-17T14:54:47.247Z","load_balancing_strategy":"primary","db_duration_s":0.18019,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1613vm-test-run-nixbot-gitlab> gitlab # [ 202.688159] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:47.249Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"e043e5562a28d2bf7ea62633","created_at":"2026-06-17T14:54:46.860Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.864Z","job_size_bytes":89,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-e043e5562a28d2bf7ea62633: start","job_status":"start","queue_duration_s":0.385149,"scheduling_latency_s":0.38518}1614vm-test-run-nixbot-gitlab> gitlab # [ 202.780689] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:47.297Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"e043e5562a28d2bf7ea62633","created_at":"2026-06-17T14:54:46.860Z","trace_propagation_headers":{"sentry-trace":"80c3844c0fe5499989d1b5987535f7fe-64da7ca4d0774d52","baggage":"sentry-trace_id=80c3844c0fe5499989d1b5987535f7fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17NC81JAFW62YC3ZGZ69G","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:46.864Z","job_size_bytes":89,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-e043e5562a28d2bf7ea62633: done: 0.048303 sec","job_status":"done","queue_duration_s":0.385149,"scheduling_latency_s":0.38518,"redis_calls":11,"redis_duration_s":0.002569,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000683,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001434,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000358146,"duration_s":0.048303,"completed_at":"2026-06-17T14:54:47.297Z","load_balancing_strategy":"primary","db_duration_s":0.015361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1615vm-test-run-nixbot-gitlab> gitlab # [ 202.882709] setup-gitlab-start[3035]: Initialized empty Git repository in /tmp/test-flake/.git/1616vm-test-run-nixbot-gitlab> gitlab # [ 202.883872] setup-gitlab-start[2717]: + git config user.name test1617vm-test-run-nixbot-gitlab> gitlab # [ 202.888342] setup-gitlab-start[2717]: + git config user.email test@example.com1618vm-test-run-nixbot-gitlab> gitlab # [ 202.891877] setup-gitlab-start[2717]: + cat1619vm-test-run-nixbot-gitlab> gitlab # [ 202.896219] setup-gitlab-start[2717]: + git add flake.nix1620vm-test-run-nixbot-gitlab> gitlab # [ 202.900210] setup-gitlab-start[2717]: + git commit -m 'initial commit'1621vm-test-run-nixbot-gitlab> gitlab # [ 202.916347] setup-gitlab-start[3040]: [master (root-commit) 69d0fad] initial commit1622vm-test-run-nixbot-gitlab> gitlab # [ 202.917430] setup-gitlab-start[3040]: 1 file changed, 10 insertions(+)1623vm-test-run-nixbot-gitlab> gitlab # [ 202.918267] setup-gitlab-start[3040]: create mode 100644 flake.nix1624vm-test-run-nixbot-gitlab> gitlab # [ 202.919162] setup-gitlab-start[2717]: + git remote add origin http://oauth2:glpat-sHOx8I_Ie75OwkI96H42_G86MQp1OjEH.01.0w1ma9rtm@gitlab/root/test-flake.git1625vm-test-run-nixbot-gitlab> gitlab # [ 202.925226] setup-gitlab-start[2717]: + git push -u origin master1626vm-test-run-nixbot-gitlab> gitlab # [ 203.133512] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:47 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 1821627vm-test-run-nixbot-gitlab> gitlab # [ 203.364230] gitaly[1386]: time="2026-06-17T14:54:47.926Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RCJWBK9ANYHG31BSZFA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:47.907" grpc.time_ms=19.243 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1628vm-test-run-nixbot-gitlab> gitlab # [ 203.374703] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:47 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 2381629vm-test-run-nixbot-gitlab> gitlab # [ 203.706189] gitaly[1386]: time="2026-06-17T14:54:48.269Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-17T14:54:58.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:48.257" grpc.time_ms=12.031 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1630vm-test-run-nixbot-gitlab> gitlab # [ 203.766096] gitaly[1386]: time="2026-06-17T14:54:48.329Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-17T14:55:18.118" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-17T14:54:48.319" grpc.time_ms=9.516 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1631vm-test-run-nixbot-gitlab> gitlab # [ 203.861575] gitaly[1386]: time="2026-06-17T14:54:48.424Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 duration_ms=284ns method=POST pid=1386 status=200 url="http://unix/api/v4/internal/allowed"1632vm-test-run-nixbot-gitlab> gitlab # [ 203.891404] gitaly[1386]: time="2026-06-17T14:54:48.454Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 duration_ms=28ns method=POST pid=1386 status=200 url="http://unix/api/v4/internal/pre_receive"1633vm-test-run-nixbot-gitlab> gitlab # [ 203.894147] gitaly[1386]: time="2026-06-17T14:54:48.455Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:48.138" grpc.time_ms=316.273 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1634vm-test-run-nixbot-gitlab> gitlab # [ 203.967120] gitaly[1386]: time="2026-06-17T14:54:48.530Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:48.529" grpc.time_ms=0.679 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1635vm-test-run-nixbot-gitlab> gitlab # [ 204.026503] gitaly[1386]: time="2026-06-17T14:54:48.589Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:48.589" grpc.time_ms=0.621 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1636vm-test-run-nixbot-gitlab> gitlab # [ 204.075777] gitaly[1386]: time="2026-06-17T14:54:48.639Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:48.638" grpc.time_ms=0.841 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1637vm-test-run-nixbot-gitlab> gitlab # [ 204.173405] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:48.735Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 69d0fad95915071fc708273468e0ca306ba5a878 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e6b3543c3a3416196e2d4e4f","created_at":"2026-06-17T14:54:48.710Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-e21a5d8f349b48da","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:48.733Z","job_size_bytes":185,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e6b3543c3a3416196e2d4e4f: start","job_status":"start","queue_duration_s":0.002812,"scheduling_latency_s":0.002853}1638vm-test-run-nixbot-gitlab> gitlab # [ 204.207910] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:48.757Z","message":"changes: \"0000000000000000000000000000000000000000 69d0fad95915071fc708273468e0ca306ba5a878 refs/heads/master\\n\""}1639vm-test-run-nixbot-gitlab> gitlab # [ 204.250795] gitaly[1386]: time="2026-06-17T14:54:48.813Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-17T14:54:58.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:48.798" grpc.time_ms=15.401 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1640vm-test-run-nixbot-gitlab> gitlab # [ 204.252245] gitaly[1386]: time="2026-06-17T14:54:48.815Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-17T14:54:58.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:48.807" grpc.time_ms=6.293 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1641vm-test-run-nixbot-gitlab> gitlab # [ 204.303931] gitaly[1386]: time="2026-06-17T14:54:48.867Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:54:58.053" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:48.857" grpc.time_ms=9.274 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1642vm-test-run-nixbot-gitlab> gitlab # [ 204.325790] gitaly[1386]: time="2026-06-17T14:54:48.889Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1573 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=6 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-17T14:54:58.021" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:48.821" grpc.time_ms=66.895 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1643vm-test-run-nixbot-gitlab> gitlab # [ 204.339233] gitaly[1386]: time="2026-06-17T14:54:48.902Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 duration_ms=221ns method=POST pid=1386 status=200 url="http://unix/api/v4/internal/post_receive"1644vm-test-run-nixbot-gitlab> gitlab # [ 204.340193] gitaly[1386]: time="2026-06-17T14:54:48.903Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:48.676" grpc.time_ms=226.364 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1645vm-test-run-nixbot-gitlab> gitlab # [ 204.344314] gitaly[1386]: time="2026-06-17T14:54:48.907Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=9 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-17T14:54:58.098" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:48.889" grpc.time_ms=17.823 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1646vm-test-run-nixbot-gitlab> gitlab # [ 204.394795] gitaly[1386]: time="2026-06-17T14:54:48.957Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=17 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-17T14:54:58.142" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:48.933" grpc.time_ms=24.5 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1647vm-test-run-nixbot-gitlab> gitlab # [ 204.400209] gitaly[1386]: time="2026-06-17T14:54:48.963Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 diskcache=aa8468aa-d49d-4969-b667-ed0fb29c5db4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:48.064" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1648vm-test-run-nixbot-gitlab> gitlab # [ 204.401180] gitaly[1386]: time="2026-06-17T14:54:48.963Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=259 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13406 command.oublock=32 command.real_time_ms=888 command.spawn_token_fork_ms=1 command.system_time_ms=186 command.user_time_ms=73 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:48.064" grpc.time_ms=898.575 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1649vm-test-run-nixbot-gitlab> gitlab # [ 204.403315] gitlab-workhorse[1289]: time="2026-06-17T14:54:48Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521650vm-test-run-nixbot-gitlab> gitlab # [ 204.404333] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:48 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 9741651vm-test-run-nixbot-gitlab> gitlab # [ 204.430432] gitaly[1386]: time="2026-06-17T14:54:48.993Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:48.993" grpc.time_ms=0.373 pid=1386 span.kind=server system=grpc1652vm-test-run-nixbot-gitlab> gitlab # [ 204.476212] setup-gitlab-start[3044]: To http://gitlab/root/test-flake.git1653vm-test-run-nixbot-gitlab> gitlab # [ 204.480125] setup-gitlab-start[3044]: * [new branch] master -> master1654vm-test-run-nixbot-gitlab> gitlab # [ 204.484395] setup-gitlab-start[3044]: branch 'master' set up to track 'origin/master'.1655vm-test-run-nixbot-gitlab> gitlab # [ 204.487727] systemd[1]: Finished setup-gitlab.service.1656vm-test-run-nixbot-gitlab> gitlab # [ 204.495122] gitaly[1386]: time="2026-06-17T14:54:49.055Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:49.053" grpc.time_ms=1.357 pid=1386 span.kind=server system=grpc1657vm-test-run-nixbot-gitlab> gitlab # [ 204.515419] systemd[1]: Starting nixbot CI service...1658vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 205.42 seconds)1659vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1660vm-test-run-nixbot-gitlab> gitlab # [ 204.521608] gitaly[1386]: time="2026-06-17T14:54:49.084Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 diskcache=56560d11-6d53-4742-b95a-a88fe5a2ae28 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-17T14:54:58.177" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:48.968" pid=1386 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1661vm-test-run-nixbot-gitlab> gitlab # [ 204.522917] gitaly[1386]: time="2026-06-17T14:54:49.085Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=62 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4717 command.oublock=8 command.real_time_ms=106 command.spawn_token_fork_ms=0 command.system_time_ms=38 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-17T14:54:58.177" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:48.968" grpc.time_ms=116.883 pid=1386 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1662vm-test-run-nixbot-gitlab> gitlab # [ 204.537953] gitaly[1386]: time="2026-06-17T14:54:49.101Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-17T14:54:59.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:49.100" grpc.time_ms=0.189 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1663vm-test-run-nixbot-gitlab> gitlab # [ 204.550504] gitaly[1386]: time="2026-06-17T14:54:49.113Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=947 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-17T14:54:59.014" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:49.104" grpc.time_ms=8.863 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1664vm-test-run-nixbot-gitlab> gitlab # [ 204.609953] gitaly[1386]: time="2026-06-17T14:54:49.173Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=337 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-17T14:55:19.066" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:49.167" grpc.time_ms=6.009 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1665vm-test-run-nixbot-gitlab> gitlab # [ 204.630738] gitaly[1386]: time="2026-06-17T14:54:49.194Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-17T14:54:59.102" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:49.193" grpc.time_ms=0.328 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1666vm-test-run-nixbot-gitlab> gitlab # [ 204.639266] gitaly[1386]: time="2026-06-17T14:54:49.202Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-17T14:54:59.107" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-17T14:54:49.198" grpc.time_ms=4.196 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1667vm-test-run-nixbot-gitlab> gitlab # [ 204.659716] gitaly[1386]: time="2026-06-17T14:54:49.222Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:54:59.126" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:49.216" grpc.time_ms=6.028 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1668vm-test-run-nixbot-gitlab> gitlab # [ 204.674094] gitaly[1386]: time="2026-06-17T14:54:49.237Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:54:59.137" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:49.227" grpc.time_ms=9.429 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1669vm-test-run-nixbot-gitlab> gitlab # [ 204.716374] gitaly[1386]: time="2026-06-17T14:54:49.279Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=21 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-17T14:55:19.146" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:49.247" grpc.time_ms=32.033 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1670vm-test-run-nixbot-gitlab> gitlab # [ 204.999742] gitaly[1386]: time="2026-06-17T14:54:49.562Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-17T14:54:59.000" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:49.561" grpc.time_ms=1.468 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1671vm-test-run-nixbot-gitlab> gitlab # [ 205.046644] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:49.609Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 69d0fad95915071fc708273468e0ca306ba5a878 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e6b3543c3a3416196e2d4e4f","created_at":"2026-06-17T14:54:48.710Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-e21a5d8f349b48da","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:48.733Z","job_size_bytes":185,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e6b3543c3a3416196e2d4e4f: done: 0.873757 sec","job_status":"done","queue_duration_s":0.002812,"scheduling_latency_s":0.002853,"gitaly_calls":14,"gitaly_duration_s":0.335714,"redis_calls":201,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.057883,"redis_read_bytes":922,"redis_write_bytes":19885,"redis_cache_calls":14,"redis_cache_duration_s":0.004208,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":32,"redis_feature_flag_duration_s":0.010875,"redis_feature_flag_read_bytes":540,"redis_feature_flag_write_bytes":4279,"redis_queues_calls":8,"redis_queues_duration_s":0.001501,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4936,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003248,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":92,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.020494,"redis_repository_cache_read_bytes":214,"redis_repository_cache_write_bytes":6392,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.017557,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":51,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":51,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":51,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.209,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.225,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.425789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001582324,"exclusive_lock_hold_duration_s":0.048308933,"duration_s":0.873757,"completed_at":"2026-06-17T14:54:49.609Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.071569,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1672vm-test-run-nixbot-gitlab> gitlab # [ 205.088191] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:49.611Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"83420ac7c7bd92392b190d62","created_at":"2026-06-17T14:54:48.909Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-f058234eeb134780","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:48.914Z","job_size_bytes":46,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-83420ac7c7bd92392b190d62: start","job_status":"start","queue_duration_s":0.696865,"scheduling_latency_s":0.696888}1673vm-test-run-nixbot-gitlab> gitlab # [ 205.093953] gitaly[1386]: time="2026-06-17T14:54:49.657Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-17T20:54:49.649" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:49.651" grpc.time_ms=5.112 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1674vm-test-run-nixbot-gitlab> gitlab # [ 205.170544] gitaly[1386]: time="2026-06-17T14:54:49.676Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-17T14:55:19.569" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:49.669" grpc.time_ms=6.692 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1675vm-test-run-nixbot-gitlab> gitlab # [ 205.173377] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:49.697Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"83420ac7c7bd92392b190d62","created_at":"2026-06-17T14:54:48.909Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-f058234eeb134780","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:48.914Z","job_size_bytes":46,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-83420ac7c7bd92392b190d62: done: 0.086593 sec","job_status":"done","queue_duration_s":0.696865,"scheduling_latency_s":0.696888,"gitaly_calls":2,"gitaly_duration_s":0.019797,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008467,"redis_read_bytes":68,"redis_write_bytes":4885,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001336,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000554,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001628,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004143,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000806,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033887,"duration_s":0.086593,"completed_at":"2026-06-17T14:54:49.697Z","load_balancing_strategy":"primary","db_duration_s":0.011979,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1676vm-test-run-nixbot-gitlab> gitlab # [ 205.271272] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:49.699Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1806179dd138d12dd637e16d","created_at":"2026-06-17T14:54:48.924Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-f058234eeb134780","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:48.928Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1806179dd138d12dd637e16d: start","job_status":"start","queue_duration_s":0.771168,"scheduling_latency_s":0.771189}1677vm-test-run-nixbot-gitlab> gitlab # [ 205.323302] gitaly[1386]: time="2026-06-17T14:54:49.886Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 diskcache=bdcb6b61-26e5-4390-a9de-4ebd53043c7a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-17T20:54:49.727" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:49.727" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1678vm-test-run-nixbot-gitlab> gitlab # [ 205.341461] gitaly[1386]: time="2026-06-17T14:54:49.886Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1283 command.oublock=0 command.real_time_ms=157 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-17T20:54:49.727" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:49.727" grpc.time_ms=158.686 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1679vm-test-run-nixbot-gitlab> gitlab # [ 205.450701] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:50.013Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1806179dd138d12dd637e16d","created_at":"2026-06-17T14:54:48.924Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-f058234eeb134780","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:48.928Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1806179dd138d12dd637e16d: done: 0.314415 sec","job_status":"done","queue_duration_s":0.771168,"scheduling_latency_s":0.771189,"gitaly_calls":1,"gitaly_duration_s":0.244017,"redis_calls":26,"redis_duration_s":0.007657,"redis_read_bytes":714,"redis_write_bytes":2673,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003228,"redis_feature_flag_read_bytes":648,"redis_feature_flag_write_bytes":1458,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002051,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000812,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001566,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":15,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.017,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.28213977,"duration_s":0.314415,"completed_at":"2026-06-17T14:54:50.013Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1680vm-test-run-nixbot-gitlab> gitlab # [ 205.549917] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:50.111Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9b43fa376f705237bf49b972","created_at":"2026-06-17T14:54:49.096Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-f058234eeb134780","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:49.098Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9b43fa376f705237bf49b972: start","job_status":"start","queue_duration_s":1.01298,"scheduling_latency_s":1.013051}1681vm-test-run-nixbot-gitlab> gitlab # [ 205.610972] gitaly[1386]: time="2026-06-17T14:54:50.173Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-17T14:55:20.063" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:50.163" grpc.time_ms=9.383 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1682vm-test-run-nixbot-gitlab> gitlab # [ 205.624417] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:50.186Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9b43fa376f705237bf49b972","created_at":"2026-06-17T14:54:49.096Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-f058234eeb134780","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:49.098Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9b43fa376f705237bf49b972: done: 0.075683 sec","job_status":"done","queue_duration_s":1.01298,"scheduling_latency_s":1.013051,"gitaly_calls":1,"gitaly_duration_s":0.01559,"redis_calls":16,"redis_duration_s":0.007338000000000001,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000516,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002553,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002697,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001572,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075683,"completed_at":"2026-06-17T14:54:50.186Z","load_balancing_strategy":"primary","db_duration_s":0.003647,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1683vm-test-run-nixbot-gitlab> gitlab # [ 205.634227] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:50.189Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"18f843b00ca45003eb7b5e4f","created_at":"2026-06-17T14:54:49.533Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-f058234eeb134780","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a3793be127e9741eb007ad75543792607348e9f660a4363e52f800a95fa45437","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:49.537Z","job_size_bytes":467,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-18f843b00ca45003eb7b5e4f: start","job_status":"start","queue_duration_s":0.651095,"scheduling_latency_s":0.651126}1684vm-test-run-nixbot-gitlab> gitlab # [ 205.662521] gitaly[1386]: time="2026-06-17T14:54:50.225Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:20.123" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:50.224" grpc.time_ms=0.616 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1685vm-test-run-nixbot-gitlab> gitlab # [ 205.681849] gitaly[1386]: time="2026-06-17T14:54:50.235Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-17T14:55:00.137" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:50.227" grpc.time_ms=7.003 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1686vm-test-run-nixbot-gitlab> gitlab # [ 205.753646] gitaly[1386]: time="2026-06-17T14:54:50.316Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-17T14:55:20.215" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:50.315" grpc.time_ms=0.925 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1687vm-test-run-nixbot-gitlab> gitlab # [ 205.763359] gitaly[1386]: time="2026-06-17T14:54:50.326Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=203 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-17T14:55:20.222" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:50.322" grpc.time_ms=3.349 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688vm-test-run-nixbot-gitlab> gitlab # [ 205.847981] gitaly[1386]: time="2026-06-17T14:54:50.411Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:00.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:50.404" grpc.time_ms=6.865 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689vm-test-run-nixbot-gitlab> gitlab # [ 205.872262] nixbot[3173]: {"ts": "2026-06-17T14:54:50.434Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1690vm-test-run-nixbot-gitlab> gitlab # [ 205.892508] nixbot[3173]: {"ts": "2026-06-17T14:54:50.455Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1691vm-test-run-nixbot-gitlab> gitlab # [ 205.910122] nixbot[3173]: {"ts": "2026-06-17T14:54:50.473Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1692vm-test-run-nixbot-gitlab> gitlab # [ 205.917478] nixbot[3173]: {"ts": "2026-06-17T14:54:50.480Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1693vm-test-run-nixbot-gitlab> gitlab # [ 205.924059] nixbot[3173]: {"ts": "2026-06-17T14:54:50.487Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1694vm-test-run-nixbot-gitlab> gitlab # [ 205.932295] nixbot[3173]: {"ts": "2026-06-17T14:54:50.495Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1695vm-test-run-nixbot-gitlab> gitlab # [ 205.938819] nixbot[3173]: {"ts": "2026-06-17T14:54:50.502Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1696vm-test-run-nixbot-gitlab> gitlab # [ 205.944626] nixbot[3173]: {"ts": "2026-06-17T14:54:50.507Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1697vm-test-run-nixbot-gitlab> gitlab # [ 205.950669] nixbot[3173]: {"ts": "2026-06-17T14:54:50.513Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1698vm-test-run-nixbot-gitlab> gitlab # [ 205.956103] nixbot[3173]: {"ts": "2026-06-17T14:54:50.519Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1699vm-test-run-nixbot-gitlab> gitlab # [ 205.961402] nixbot[3173]: {"ts": "2026-06-17T14:54:50.524Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1700vm-test-run-nixbot-gitlab> gitlab # [ 205.967773] nixbot[3173]: {"ts": "2026-06-17T14:54:50.530Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1701vm-test-run-nixbot-gitlab> gitlab # [ 205.970575] nixbot[3173]: {"ts": "2026-06-17T14:54:50.533Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1702vm-test-run-nixbot-gitlab> gitlab # [ 205.974264] nixbot[3173]: {"ts": "2026-06-17T14:54:50.537Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1703vm-test-run-nixbot-gitlab> gitlab # [ 205.982601] nixbot[3173]: {"ts": "2026-06-17T14:54:50.545Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1704vm-test-run-nixbot-gitlab> gitlab # [ 205.988360] nixbot[3173]: {"ts": "2026-06-17T14:54:50.551Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1705vm-test-run-nixbot-gitlab> gitlab # [ 205.994085] nixbot[3173]: {"ts": "2026-06-17T14:54:50.557Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1706vm-test-run-nixbot-gitlab> gitlab # [ 205.996178] nixbot[3173]: {"ts": "2026-06-17T14:54:50.559Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1707vm-test-run-nixbot-gitlab> gitlab # [ 206.300821] nixbot[3173]: INFO: Started server process [3173]1708vm-test-run-nixbot-gitlab> gitlab # [ 206.304697] nixbot[3173]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1709vm-test-run-nixbot-gitlab> gitlab # [ 206.311420] nixbot[3173]: INFO: ::ffff:127.0.0.1:55242 - "GET /health HTTP/1.1" 200 OK1710vm-test-run-nixbot-gitlab> gitlab # [ 206.317824] systemd[1]: Started nixbot CI service.1711vm-test-run-nixbot-gitlab> gitlab # [ 206.322726] systemd[1]: Reached target Multi-User System.1712vm-test-run-nixbot-gitlab> gitlab # [ 206.326135] systemd[1]: Startup finished in 1.385s (kernel) + 3.909s (initrd) + 3min 21.026s (userspace) = 3min 26.320s.1713vm-test-run-nixbot-gitlab> gitlab # [ 206.411762] gitaly[1386]: time="2026-06-17T14:54:50.975Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-17T14:55:20.862" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:50.963" grpc.time_ms=11.788 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1714vm-test-run-nixbot-gitlab> gitlab # [ 206.425886] gitaly[1386]: time="2026-06-17T14:54:50.988Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17RNQ1ZAEYNMYWPK4JBB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-17T14:55:20.878" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:50.979" grpc.time_ms=9.554 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1715vm-test-run-nixbot-gitlab> gitlab # [ 206.459672] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-06-17T14:54:51.022Z","class":"CreatePipelineWorker","correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"69d0fad95915071fc708273468e0ca306ba5a878","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"69d0fad95915071fc708273468e0ca306ba5a878","ref":"refs/heads/master"},"retry":0}1716vm-test-run-nixbot-gitlab> gitlab # [ 206.465896] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:51.028Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"18f843b00ca45003eb7b5e4f","created_at":"2026-06-17T14:54:49.533Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-f058234eeb134780","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a3793be127e9741eb007ad75543792607348e9f660a4363e52f800a95fa45437","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:49.537Z","job_size_bytes":467,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-18f843b00ca45003eb7b5e4f: done: 0.839963 sec","job_status":"done","queue_duration_s":0.651095,"scheduling_latency_s":0.651126,"gitaly_calls":7,"gitaly_duration_s":0.083989,"redis_calls":53,"redis_duration_s":0.014376,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.001628,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.005306,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004145,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.6695,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000380775,"duration_s":0.839963,"completed_at":"2026-06-17T14:54:51.028Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053905,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1717vm-test-run-nixbot-gitlab> gitlab # [ 206.501177] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:51.035Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"4de1ec125e744f9d0835418b","created_at":"2026-06-17T14:54:49.684Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-fd5d3a9fd7cf4778","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:49.688Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-4de1ec125e744f9d0835418b: start","job_status":"start","queue_duration_s":1.346328,"scheduling_latency_s":1.34635}1718vm-test-run-nixbot-gitlab> gitlab # [ 206.527354] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:51.085Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"4de1ec125e744f9d0835418b","created_at":"2026-06-17T14:54:49.684Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-fd5d3a9fd7cf4778","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:49.688Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-4de1ec125e744f9d0835418b: done: 0.050386 sec","job_status":"done","queue_duration_s":1.346328,"scheduling_latency_s":1.34635,"redis_calls":19,"redis_duration_s":0.015292,"redis_read_bytes":21,"redis_write_bytes":4290,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003279,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.001688,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2261,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.008314,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.011211760000000001,"duration_s":0.050386,"completed_at":"2026-06-17T14:54:51.085Z","load_balancing_strategy":"primary","db_duration_s":0.006462,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1719vm-test-run-nixbot-gitlab> gitlab # [ 206.625311] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:51.089Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"65ee2250adb4d589625db278","created_at":"2026-06-17T14:54:51.067Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-ac3e451002b94eb9","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:51.074Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-65ee2250adb4d589625db278: start","job_status":"start","queue_duration_s":0.014786,"scheduling_latency_s":0.014806}1720vm-test-run-nixbot-gitlab> gitlab # [ 206.626900] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:51.156Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"65ee2250adb4d589625db278","created_at":"2026-06-17T14:54:51.067Z","trace_propagation_headers":{"sentry-trace":"31b6e5a6ad7b42778b66f75b8fa1eb5c-ac3e451002b94eb9","baggage":"sentry-trace_id=31b6e5a6ad7b42778b66f75b8fa1eb5c,sentry-release=Unknown"},"correlation_id":"01KVB17RNQ1ZAEYNMYWPK4JBB7","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:51.074Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-65ee2250adb4d589625db278: done: 0.067881 sec","job_status":"done","queue_duration_s":0.014786,"scheduling_latency_s":0.014806,"redis_calls":11,"redis_duration_s":0.006214,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003307,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00064,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349485,"duration_s":0.067881,"completed_at":"2026-06-17T14:54:51.156Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1721vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 2.29 seconds)1722vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1723vm-test-run-nixbot-gitlab> gitlab # [ 206.918424] nixbot[3173]: INFO: ::ffff:127.0.0.1:55248 - "GET /health HTTP/1.1" 200 OK1724vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1725vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1726vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1727vm-test-run-nixbot-gitlab> gitlab # [ 207.031451] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 781728vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1729vm-test-run-nixbot-gitlab> []1730vm-test-run-nixbot-gitlab> gitlab # [ 207.094510] gitaly[1386]: time="2026-06-17T14:54:51.657Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB17VFRXVKHVRYVW2RC6Z0C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-17T14:55:21.442" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:51.642" grpc.time_ms=14.693 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1731vm-test-run-nixbot-gitlab> gitlab # [ 207.437689] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 11271732vm-test-run-nixbot-gitlab> gitlab # [ 207.449247] nixbot[3173]: {"ts": "2026-06-17T14:54:52.012Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1733vm-test-run-nixbot-gitlab> gitlab # [ 207.496611] nixbot[3173]: {"ts": "2026-06-17T14:54:52.059Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1734vm-test-run-nixbot-gitlab> gitlab # [ 207.589936] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 671735vm-test-run-nixbot-gitlab> gitlab # [ 207.600324] nixbot[3173]: {"ts": "2026-06-17T14:54:52.153Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1736vm-test-run-nixbot-gitlab> gitlab # [ 207.610931] nixbot[3173]: {"ts": "2026-06-17T14:54:52.155Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1737vm-test-run-nixbot-gitlab> gitlab # [ 207.690625] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 951738vm-test-run-nixbot-gitlab> gitlab # [ 207.694290] nixbot[3173]: {"ts": "2026-06-17T14:54:52.254Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1739vm-test-run-nixbot-gitlab> gitlab # [ 207.973585] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 2641740vm-test-run-nixbot-gitlab> gitlab # [ 207.984789] nixbot[3173]: {"ts": "2026-06-17T14:54:52.538Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1741vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1742vm-test-run-nixbot-gitlab> gitlab # [ 208.061783] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 721743vm-test-run-nixbot-gitlab> gitlab # [ 208.063663] nixbot[3173]: {"ts": "2026-06-17T14:54:52.626Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1744vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds)1745vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-17T14:54:52.236Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1746vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.21 seconds)1747vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1748vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1749vm-test-run-nixbot-gitlab> gitlab # [ 208.125688] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 451750vm-test-run-nixbot-gitlab> gitlab # [ 208.151316] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 711751vm-test-run-nixbot-gitlab> gitlab # [ 208.160258] nixbot[3173]: {"ts": "2026-06-17T14:54:52.715Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1752vm-test-run-nixbot-gitlab> gitlab # [ 208.212096] gitaly[1386]: time="2026-06-17T14:54:52.774Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17X9G5QHHRVEQ4JGRJTZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-17T14:55:22.063" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:52.764" grpc.time_ms=10.416 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753vm-test-run-nixbot-gitlab> gitlab # [ 208.233445] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 371754vm-test-run-nixbot-gitlab> gitlab # [ 208.268601] gitaly[1386]: time="2026-06-17T14:54:52.831Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17X9G5QHHRVEQ4JGRJTZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:22.129" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:52.830" grpc.time_ms=1.374 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755vm-test-run-nixbot-gitlab> gitlab # [ 208.297707] gitaly[1386]: time="2026-06-17T14:54:52.860Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XBX4WQXK4SSM269NDAZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:52.852" grpc.time_ms=7.868 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1756vm-test-run-nixbot-gitlab> gitlab # [ 208.300522] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 651757vm-test-run-nixbot-gitlab> gitlab # [ 208.305497] gitaly[1386]: time="2026-06-17T14:54:52.868Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17X9G5QHHRVEQ4JGRJTZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-17T14:55:02.008" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:52.858" grpc.time_ms=9.852 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758vm-test-run-nixbot-gitlab> gitlab # [ 208.345994] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1881759vm-test-run-nixbot-gitlab> gitlab # [ 208.350255] nixbot[3173]: {"ts": "2026-06-17T14:54:52.910Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1760vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1761vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1762vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1763vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1764vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done.1765vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1766vm-test-run-nixbot-gitlab> gitlab # [ 208.419063] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 681767vm-test-run-nixbot-gitlab> gitlab # [ 208.424241] nixbot[3173]: {"ts": "2026-06-17T14:54:52.987Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1768vm-test-run-nixbot-gitlab> gitlab # [ 208.428856] nixbot[3173]: {"ts": "2026-06-17T14:54:52.991Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1769vm-test-run-nixbot-gitlab> gitlab # [ 208.522063] gitaly[1386]: time="2026-06-17T14:54:53.085Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XK10F77YJSK4KWRGBD6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.069" grpc.time_ms=15.177 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1770vm-test-run-nixbot-gitlab> gitlab # [ 208.524883] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:53 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 611771vm-test-run-nixbot-gitlab> gitlab # [ 208.573408] gitaly[1386]: time="2026-06-17T14:54:53.136Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-17T14:55:03.003" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:53.125" grpc.time_ms=10.876 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772vm-test-run-nixbot-gitlab> gitlab # [ 208.618320] gitaly[1386]: time="2026-06-17T14:54:53.181Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XN19J35P5SKP9XFRR27 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.170" pid=1386 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1773vm-test-run-nixbot-gitlab> gitlab # [ 208.619572] gitaly[1386]: time="2026-06-17T14:54:53.181Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XN19J35P5SKP9XFRR27 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.170" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1774vm-test-run-nixbot-gitlab> gitlab # [ 208.620392] gitaly[1386]: time="2026-06-17T14:54:53.182Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XN19J35P5SKP9XFRR27 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.170" grpc.time_ms=11.261 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1775vm-test-run-nixbot-gitlab> gitlab # [ 208.622359] gitlab-workhorse[1289]: time="2026-06-17T14:54:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491776vm-test-run-nixbot-gitlab> gitlab # [ 208.625602] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:53 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 931777vm-test-run-nixbot-gitlab> gitlab # [ 208.692807] gitaly[1386]: time="2026-06-17T14:54:53.253Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XR3T4T0XVXC7XS98K8G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.231" pid=1386 remote_ip="2001:db8:1::1" request_sha=c9fc1a0092218228bafef7bd337990b3dc025799 response_bytes=378 span.kind=server system=grpc1778vm-test-run-nixbot-gitlab> gitlab # [ 208.694112] gitlab-workhorse[1289]: time="2026-06-17T14:54:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3781779vm-test-run-nixbot-gitlab> gitlab # [ 208.697101] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:53 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 378 "" "git/2.54.0" 681780vm-test-run-nixbot-gitlab> gitlab # [ 208.697970] gitaly[1386]: time="2026-06-17T14:54:53.253Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1228 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=5 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XR3T4T0XVXC7XS98K8G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.231" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1781vm-test-run-nixbot-gitlab> gitlab # [ 208.699191] gitaly[1386]: time="2026-06-17T14:54:53.257Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1228 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=5 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XR3T4T0XVXC7XS98K8G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.231" grpc.time_ms=22.988 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1782vm-test-run-nixbot-gitlab> gitlab # [ 208.763892] gitaly[1386]: time="2026-06-17T14:54:53.327Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=68 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=0 command.real_time_ms=72 command.spawn_token_fork_ms=8 command.system_time_ms=51 command.user_time_ms=17 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-17T14:55:23.081" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-17T14:54:53.181" grpc.time_ms=145.261 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783vm-test-run-nixbot-gitlab> gitlab # [ 208.797653] gitaly[1386]: time="2026-06-17T14:54:53.360Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KVB17XG39NE02GMG6R8QBEJX duration_ms=309ns method=POST pid=1386 status=200 url="http://unix/api/v4/internal/allowed"1784vm-test-run-nixbot-gitlab> gitlab # [ 208.827111] gitaly[1386]: time="2026-06-17T14:54:53.388Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KVB17XG39NE02GMG6R8QBEJX duration_ms=25ns method=POST pid=1386 status=200 url="http://unix/api/v4/internal/pre_receive"1785vm-test-run-nixbot-gitlab> gitlab # [ 208.828217] gitaly[1386]: time="2026-06-17T14:54:53.389Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:53.050" grpc.time_ms=338.468 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1786vm-test-run-nixbot-gitlab> gitlab # [ 208.878118] gitaly[1386]: time="2026-06-17T14:54:53.437Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=11 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XXSSYHBNDB7X6HJ4KJM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.419" grpc.time_ms=18.024 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1787vm-test-run-nixbot-gitlab> gitlab # [ 208.879329] gitaly[1386]: time="2026-06-17T14:54:53.440Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:53.439" grpc.time_ms=0.51 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1788vm-test-run-nixbot-gitlab> gitlab # [ 208.882994] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:53 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 761789vm-test-run-nixbot-gitlab> gitlab # [ 208.947221] gitaly[1386]: time="2026-06-17T14:54:53.510Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y08VDTBZD49TDWPRN4S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.495" pid=1386 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1790vm-test-run-nixbot-gitlab> gitlab # [ 208.948745] gitaly[1386]: time="2026-06-17T14:54:53.510Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y08VDTBZD49TDWPRN4S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.495" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1791vm-test-run-nixbot-gitlab> gitlab # [ 208.949812] gitlab-workhorse[1289]: time="2026-06-17T14:54:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671792vm-test-run-nixbot-gitlab> gitlab # [ 208.951003] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:53 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 631793vm-test-run-nixbot-gitlab> gitlab # [ 208.951571] gitaly[1386]: time="2026-06-17T14:54:53.511Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y08VDTBZD49TDWPRN4S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:53.495" grpc.time_ms=15.806 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1794vm-test-run-nixbot-gitlab> gitlab # [ 208.958896] gitaly[1386]: time="2026-06-17T14:54:53.522Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:53.521" grpc.time_ms=0.486 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1795vm-test-run-nixbot-gitlab> gitlab # [ 209.057287] gitaly[1386]: time="2026-06-17T14:54:53.620Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:53.619" grpc.time_ms=0.454 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1796vm-test-run-nixbot-gitlab> gitlab # [ 209.113424] gitaly[1386]: time="2026-06-17T14:54:53.676Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:23.074" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:53.675" grpc.time_ms=0.654 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1797vm-test-run-nixbot-gitlab> gitlab # [ 209.136477] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:53.699Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","69d0fad95915071fc708273468e0ca306ba5a878 14b3955733a0a15351b056a59a904cda74dfdf9d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"1abec557fe6592c2eb0f0305","created_at":"2026-06-17T14:54:53.690Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-84c6adc3013c483a","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:53.692Z","job_size_bytes":185,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-1abec557fe6592c2eb0f0305: start","job_status":"start","queue_duration_s":0.006917,"scheduling_latency_s":0.006958}1798vm-test-run-nixbot-gitlab> gitlab # [ 209.163716] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:53.724Z","message":"changes: \"69d0fad95915071fc708273468e0ca306ba5a878 14b3955733a0a15351b056a59a904cda74dfdf9d refs/heads/master\\n\""}1799vm-test-run-nixbot-gitlab> gitlab # [ 209.168134] gitaly[1386]: time="2026-06-17T14:54:53.719Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=6 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-17T14:55:23.110" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:53.710" grpc.time_ms=8.763 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800vm-test-run-nixbot-gitlab> gitlab # [ 209.190697] gitaly[1386]: time="2026-06-17T14:54:53.740Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KVB17XG39NE02GMG6R8QBEJX duration_ms=81ns method=POST pid=1386 status=200 url="http://unix/api/v4/internal/post_receive"1801vm-test-run-nixbot-gitlab> gitlab # [ 209.195944] gitaly[1386]: time="2026-06-17T14:54:53.741Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:53.658" grpc.time_ms=82.674 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1802vm-test-run-nixbot-gitlab> gitlab # [ 209.213861] gitaly[1386]: time="2026-06-17T14:54:53.776Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=40 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:03.284" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:53.734" grpc.time_ms=41.822 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803vm-test-run-nixbot-gitlab> gitlab # [ 209.214962] gitaly[1386]: time="2026-06-17T14:54:53.776Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:03.192" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:53.752" grpc.time_ms=23.454 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804vm-test-run-nixbot-gitlab> gitlab # [ 209.226651] gitaly[1386]: time="2026-06-17T14:54:53.789Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-17T14:55:03.221" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:53.782" grpc.time_ms=7.552 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805vm-test-run-nixbot-gitlab> gitlab # [ 209.238664] gitaly[1386]: time="2026-06-17T14:54:53.801Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX diskcache=776ea9a7-7188-4faf-b9aa-d62c0defd386 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:52.994" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1806vm-test-run-nixbot-gitlab> gitlab # [ 209.242322] gitaly[1386]: time="2026-06-17T14:54:53.803Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=239 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13281 command.oublock=32 command.real_time_ms=795 command.spawn_token_fork_ms=3 command.system_time_ms=197 command.user_time_ms=42 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:52.994" grpc.time_ms=807.783 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1807vm-test-run-nixbot-gitlab> gitlab # [ 209.245785] gitlab-workhorse[1289]: time="2026-06-17T14:54:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521808vm-test-run-nixbot-gitlab> gitlab # [ 209.248450] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:53 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 8731809vm-test-run-nixbot-gitlab> gitlab # [ 209.251091] gitaly[1386]: time="2026-06-17T14:54:53.805Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-17T14:55:03.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:53.787" grpc.time_ms=17.394 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1810vm-test-run-nixbot-gitlab> gitlab # [ 209.273510] gitaly[1386]: time="2026-06-17T14:54:53.816Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-17T14:55:03.248" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:53.809" grpc.time_ms=7.123 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1811vm-test-run-nixbot-gitlab> gitlab # [ 209.294501] gitaly[1386]: time="2026-06-17T14:54:53.829Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-17T14:55:23.720" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:53.820" grpc.time_ms=8.149 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812vm-test-run-nixbot-gitlab> gitlab # [ 209.319236] gitaly[1386]: time="2026-06-17T14:54:53.857Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=16 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-17T14:55:03.278" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-17T14:54:53.838" grpc.time_ms=18.529 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813vm-test-run-nixbot-gitlab> gitlab # [ 209.340823] gitaly[1386]: time="2026-06-17T14:54:53.873Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:03.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:53.864" grpc.time_ms=8.316 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1815vm-test-run-nixbot-gitlab> gitlab # 69d0fad..14b3955 master -> master1816vm-test-run-nixbot-gitlab> gitlab # [ 209.377643] gitaly[1386]: time="2026-06-17T14:54:53.940Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-17T14:55:03.159" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:53.940" grpc.time_ms=0.263 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817vm-test-run-nixbot-gitlab> gitlab # [ 209.436424] gitaly[1386]: time="2026-06-17T14:54:53.999Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=47 command.spawn_token_fork_ms=42 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-17T14:55:03.169" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:53.950" grpc.time_ms=49.047 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.35 seconds)1819vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1820vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1821vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=14b3955733a0a15351b056a59a904cda74dfdf9d'1822vm-test-run-nixbot-gitlab> gitlab # [ 209.491743] gitaly[1386]: time="2026-06-17T14:54:54.054Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:54.054" grpc.time_ms=0.503 pid=1386 span.kind=server system=grpc1823vm-test-run-nixbot-gitlab> gitlab # [ 209.516709] gitaly[1386]: time="2026-06-17T14:54:54.079Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-17T14:55:04.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:54.075" grpc.time_ms=3.914 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1824vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=14b3955733a0a15351b056a59a904cda74dfdf9d', in 0.04 seconds)1825vm-test-run-nixbot-gitlab> []1826vm-test-run-nixbot-gitlab> gitlab # [ 209.538117] nixbot[3173]: INFO: ::ffff:127.0.0.1:55258 - "GET /api/repos/gitlab/root/test-flake/builds?commit=14b3955733a0a15351b056a59a904cda74dfdf9d HTTP/1.1" 200 OK1827vm-test-run-nixbot-gitlab> gitlab # [ 209.541808] gitaly[1386]: time="2026-06-17T14:54:54.101Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:54.099" grpc.time_ms=1.522 pid=1386 span.kind=server system=grpc1828vm-test-run-nixbot-gitlab> gitlab # [ 209.544646] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:54.107Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","69d0fad95915071fc708273468e0ca306ba5a878 14b3955733a0a15351b056a59a904cda74dfdf9d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"1abec557fe6592c2eb0f0305","created_at":"2026-06-17T14:54:53.690Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-84c6adc3013c483a","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:54:53.692Z","job_size_bytes":185,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-1abec557fe6592c2eb0f0305: done: 0.408618 sec","job_status":"done","queue_duration_s":0.006917,"scheduling_latency_s":0.006958,"gitaly_calls":7,"gitaly_duration_s":0.102886,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.031871,"redis_read_bytes":350,"redis_write_bytes":13070,"redis_cache_calls":9,"redis_cache_duration_s":0.002263,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001716,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.00153,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6770,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.007318,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.009282,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.009762,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.062,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.062,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.183759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001370565,"exclusive_lock_hold_duration_s":0.042826113,"duration_s":0.408618,"completed_at":"2026-06-17T14:54:54.107Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036954,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1829vm-test-run-nixbot-gitlab> gitlab # [ 209.568198] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:54.109Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"53c5cdb9088c178b4847dafa","created_at":"2026-06-17T14:54:53.791Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-537eb4438a3543e0","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:53.793Z","job_size_bytes":46,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-53c5cdb9088c178b4847dafa: start","job_status":"start","queue_duration_s":0.315511,"scheduling_latency_s":0.315589}1830vm-test-run-nixbot-gitlab> gitlab # [ 209.594122] gitaly[1386]: time="2026-06-17T14:54:54.114Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH diskcache=bb3a85e2-25b3-48a8-b8f5-6a72436d107d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-17T14:55:04.004" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:54.004" pid=1386 ref=refs/keep-around/69d0fad95915071fc708273468e0ca306ba5a878 remote_ip="2001:db8:1::1" revision=69d0fad95915071fc708273468e0ca306ba5a878 span.kind=server system=grpc username=root1831vm-test-run-nixbot-gitlab> gitlab # [ 209.612231] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:54.134Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"53c5cdb9088c178b4847dafa","created_at":"2026-06-17T14:54:53.791Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-537eb4438a3543e0","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:53.793Z","job_size_bytes":46,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-53c5cdb9088c178b4847dafa: done: 0.025408 sec","job_status":"done","queue_duration_s":0.315511,"scheduling_latency_s":0.315589,"redis_calls":9,"redis_duration_s":0.004961,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000239,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025408,"completed_at":"2026-06-17T14:54:54.134Z","load_balancing_strategy":"primary","db_duration_s":0.003981,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1832vm-test-run-nixbot-gitlab> gitlab # [ 209.693874] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:54.136Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"47c89859df2cd4249e4bc4ac","created_at":"2026-06-17T14:54:53.800Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-537eb4438a3543e0","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:53.804Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-47c89859df2cd4249e4bc4ac: start","job_status":"start","queue_duration_s":0.332141,"scheduling_latency_s":0.332165}1833vm-test-run-nixbot-gitlab> gitlab # [ 209.708889] gitaly[1386]: time="2026-06-17T14:54:54.115Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=52 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5148 command.oublock=8 command.real_time_ms=99 command.spawn_token_fork_ms=3 command.system_time_ms=35 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-17T14:55:04.004" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:54.004" grpc.time_ms=110.187 pid=1386 ref=refs/keep-around/69d0fad95915071fc708273468e0ca306ba5a878 remote_ip="2001:db8:1::1" revision=69d0fad95915071fc708273468e0ca306ba5a878 span.kind=server system=grpc username=root1834vm-test-run-nixbot-gitlab> gitlab # [ 209.732922] nixbot[3173]: {"ts": "2026-06-17T14:54:54.247Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/69d0fad95915071fc708273468e0ca306ba5a878 \"HTTP/1.1 201 Created\""}1835vm-test-run-nixbot-gitlab> gitlab # [ 209.737127] nixbot[3173]: {"ts": "2026-06-17T14:54:54.254Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-69d0fad95915-5dd44eee", "argv0": "bwrap"}1836vm-test-run-nixbot-gitlab> gitlab # [ 209.741132] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:54.213Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"47c89859df2cd4249e4bc4ac","created_at":"2026-06-17T14:54:53.800Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-537eb4438a3543e0","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:53.804Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-47c89859df2cd4249e4bc4ac: done: 0.077208 sec","job_status":"done","queue_duration_s":0.332141,"scheduling_latency_s":0.332165,"gitaly_calls":1,"gitaly_duration_s":0.043566,"redis_calls":12,"redis_duration_s":0.004965000000000001,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002906,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000673,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001386,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.058417201,"duration_s":0.077208,"completed_at":"2026-06-17T14:54:54.213Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1837vm-test-run-nixbot-gitlab> gitlab # [ 209.814320] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:54.218Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"969019bc26b9e8849b346624","created_at":"2026-06-17T14:54:53.956Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-537eb4438a3543e0","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ef8a4bc67c2e2e10730804d7f784a97a6d4bd09a7c2c8d2aebdf661ca737bf42","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:53.958Z","job_size_bytes":467,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-969019bc26b9e8849b346624: start","job_status":"start","queue_duration_s":0.259961,"scheduling_latency_s":0.259983}1838vm-test-run-nixbot-gitlab> gitlab # [ 209.904318] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:54 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/69d0fad95915071fc708273468e0ca306ba5a878 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 6201839vm-test-run-nixbot-gitlab> gitlab # [ 209.905762] gitaly[1386]: time="2026-06-17T14:54:54.122Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:24.018" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:54.118" grpc.time_ms=2.806 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840vm-test-run-nixbot-gitlab> gitlab # [ 209.906883] gitaly[1386]: time="2026-06-17T14:54:54.195Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX diskcache=41f97ae4-1e4f-40df-84b7-114f2de2f914 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-17T20:54:54.153" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:54.154" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1841vm-test-run-nixbot-gitlab> gitlab # [ 209.908094] gitaly[1386]: time="2026-06-17T14:54:54.195Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1652 command.oublock=0 command.real_time_ms=47 command.spawn_token_fork_ms=8 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-17T20:54:54.153" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:54.154" grpc.time_ms=41.262 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842vm-test-run-nixbot-gitlab> gitlab # [ 209.908917] gitaly[1386]: time="2026-06-17T14:54:54.264Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:24.061" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:54.262" grpc.time_ms=1.348 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1843vm-test-run-nixbot-gitlab> gitlab # [ 209.925248] gitaly[1386]: time="2026-06-17T14:54:54.274Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-17T14:55:04.195" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:54.266" grpc.time_ms=7.642 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1844vm-test-run-nixbot-gitlab> gitlab # [ 209.946087] gitaly[1386]: time="2026-06-17T14:54:54.406Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-17T14:55:24.204" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:54.405" grpc.time_ms=0.869 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845vm-test-run-nixbot-gitlab> gitlab # [ 209.966476] gitaly[1386]: time="2026-06-17T14:54:54.412Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=151 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-17T14:55:24.209" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:54.409" grpc.time_ms=2.673 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1846vm-test-run-nixbot-gitlab> gitlab # [ 210.048200] gitaly[1386]: time="2026-06-17T14:54:54.454Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:04.047" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:54.447" grpc.time_ms=7.038 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1847vm-test-run-nixbot-gitlab> gitlab # [ 210.151749] systemd[1]: Started Nix Daemon.1848vm-test-run-nixbot-gitlab> gitlab # [ 210.202889] nix-daemon[3447]: accepted connection from pid 3445, user nixbot1849vm-test-run-nixbot-gitlab> gitlab # [ 210.207458] nix-daemon[3447]: accepted connection from pid 3446, user nixbot1850vm-test-run-nixbot-gitlab> gitlab # [ 210.451259] gitaly[1386]: time="2026-06-17T14:54:55.014Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=965 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-17T14:55:24.793" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:54.994" grpc.time_ms=19.43 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1851vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=14b3955733a0a15351b056a59a904cda74dfdf9d'1852vm-test-run-nixbot-gitlab> gitlab # [ 210.526393] gitaly[1386]: time="2026-06-17T14:54:55.089Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB17XG39NE02GMG6R8QBEJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-17T14:55:25.067" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:55.068" grpc.time_ms=20.52 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1853vm-test-run-nixbot-gitlab> gitlab # [ 210.548259] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-06-17T14:54:55.110Z","class":"CreatePipelineWorker","correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"69d0fad95915071fc708273468e0ca306ba5a878","after":"14b3955733a0a15351b056a59a904cda74dfdf9d","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"14b3955733a0a15351b056a59a904cda74dfdf9d","ref":"refs/heads/master"},"retry":0}1854vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=14b3955733a0a15351b056a59a904cda74dfdf9d', in 0.05 seconds)1855vm-test-run-nixbot-gitlab> []1856vm-test-run-nixbot-gitlab> gitlab # [ 210.559855] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"969019bc26b9e8849b346624","created_at":"2026-06-17T14:54:53.956Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-537eb4438a3543e0","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ef8a4bc67c2e2e10730804d7f784a97a6d4bd09a7c2c8d2aebdf661ca737bf42","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:53.958Z","job_size_bytes":467,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-969019bc26b9e8849b346624: done: 0.904312 sec","job_status":"done","queue_duration_s":0.259961,"scheduling_latency_s":0.259983,"gitaly_calls":7,"gitaly_duration_s":0.129397,"redis_calls":33,"redis_duration_s":0.011106,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001975,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001644,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.005332,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002155,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.299,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.193,"db_ci_duration_s":0.106,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.460454,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.904312,"completed_at":"2026-06-17T14:54:55.122Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.298679,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1857vm-test-run-nixbot-gitlab> gitlab # [ 210.597515] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.126Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"de1a6a65-096d-4cf5-a32f-60219ce9df1b\"}"],"class":"WebHookWorker","jid":"a6e5203ecfe58dd3cf4f6c88","created_at":"2026-06-17T14:54:54.052Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-537eb4438a3543e0","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:eadc137ab9cee660bbdad648de8d6bfe504b6b7a842e8cd37c14b09d2bf23ee6","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:54.064Z","job_size_bytes":1749,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a6e5203ecfe58dd3cf4f6c88: start","job_status":"start","queue_duration_s":1.061542,"scheduling_latency_s":1.061602}1858vm-test-run-nixbot-gitlab> gitlab # [ 210.599375] nixbot[3173]: INFO: ::ffff:127.0.0.1:55266 - "GET /api/repos/gitlab/root/test-flake/builds?commit=14b3955733a0a15351b056a59a904cda74dfdf9d HTTP/1.1" 200 OK1859vm-test-run-nixbot-gitlab> gitlab # [ 210.653115] nixbot[3173]: INFO: ::ffff:127.0.0.1:55282 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1860vm-test-run-nixbot-gitlab> gitlab # [ 210.678813] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.241Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"de1a6a65-096d-4cf5-a32f-60219ce9df1b\"}"],"class":"WebHookWorker","jid":"a6e5203ecfe58dd3cf4f6c88","created_at":"2026-06-17T14:54:54.052Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-537eb4438a3543e0","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:eadc137ab9cee660bbdad648de8d6bfe504b6b7a842e8cd37c14b09d2bf23ee6","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:54.064Z","job_size_bytes":1749,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a6e5203ecfe58dd3cf4f6c88: done: 0.115456 sec","job_status":"done","queue_duration_s":1.061542,"scheduling_latency_s":1.061602,"redis_calls":19,"redis_duration_s":0.006178,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001493,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000408,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002873,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001404,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.010615316000013308,"cpu_s":0.081715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000904864,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.115456,"completed_at":"2026-06-17T14:54:55.241Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1861vm-test-run-nixbot-gitlab> gitlab # [ 210.685288] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.245Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9d950c87f192c26269305a1b","created_at":"2026-06-17T14:54:54.124Z","trace_propagation_headers":{"sentry-trace":"ff992e5849c246cf91cb075205523472-3016d22981cf486d","baggage":"sentry-trace_id=ff992e5849c246cf91cb075205523472,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17Y5S8W5AYPQGPC5KHKZH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:54.132Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9d950c87f192c26269305a1b: start","job_status":"start","queue_duration_s":1.113222,"scheduling_latency_s":1.113245}1862vm-test-run-nixbot-gitlab> gitlab # [ 210.727716] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.290Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9d950c87f192c26269305a1b","created_at":"2026-06-17T14:54:54.124Z","trace_propagation_headers":{"sentry-trace":"ff992e5849c246cf91cb075205523472-3016d22981cf486d","baggage":"sentry-trace_id=ff992e5849c246cf91cb075205523472,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17Y5S8W5AYPQGPC5KHKZH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:54.132Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9d950c87f192c26269305a1b: done: 0.044511 sec","job_status":"done","queue_duration_s":1.113222,"scheduling_latency_s":1.113245,"redis_calls":18,"redis_duration_s":0.003989,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002151,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034781,"duration_s":0.044511,"completed_at":"2026-06-17T14:54:55.290Z","load_balancing_strategy":"primary","db_duration_s":0.010397,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1863vm-test-run-nixbot-gitlab> gitlab # [ 210.785226] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.292Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"606741401c6b6d94cf868443","created_at":"2026-06-17T14:54:54.133Z","trace_propagation_headers":{"sentry-trace":"ff992e5849c246cf91cb075205523472-3016d22981cf486d","baggage":"sentry-trace_id=ff992e5849c246cf91cb075205523472,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17Y5S8W5AYPQGPC5KHKZH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:54.137Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-606741401c6b6d94cf868443: start","job_status":"start","queue_duration_s":1.154618,"scheduling_latency_s":1.15464}1864vm-test-run-nixbot-gitlab> gitlab # [ 210.813248] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.349Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"606741401c6b6d94cf868443","created_at":"2026-06-17T14:54:54.133Z","trace_propagation_headers":{"sentry-trace":"ff992e5849c246cf91cb075205523472-3016d22981cf486d","baggage":"sentry-trace_id=ff992e5849c246cf91cb075205523472,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17Y5S8W5AYPQGPC5KHKZH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:54.137Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-606741401c6b6d94cf868443: done: 0.05703 sec","job_status":"done","queue_duration_s":1.154618,"scheduling_latency_s":1.15464,"redis_calls":15,"redis_duration_s":0.003559,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000686,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001554,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001319,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000330489,"duration_s":0.05703,"completed_at":"2026-06-17T14:54:55.349Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01437,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1865vm-test-run-nixbot-gitlab> gitlab # [ 210.898233] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.350Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5b1c4f6f9102426f2a3dc437","created_at":"2026-06-17T14:54:54.230Z","trace_propagation_headers":{"sentry-trace":"ff992e5849c246cf91cb075205523472-3016d22981cf486d","baggage":"sentry-trace_id=ff992e5849c246cf91cb075205523472,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17Y5S8W5AYPQGPC5KHKZH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:54.235Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5b1c4f6f9102426f2a3dc437: start","job_status":"start","queue_duration_s":1.115579,"scheduling_latency_s":1.1156}1866vm-test-run-nixbot-gitlab> gitlab # [ 210.911382] gitaly[1386]: time="2026-06-17T14:54:55.436Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-17T14:55:25.234" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:55.435" grpc.time_ms=0.899 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867vm-test-run-nixbot-gitlab> gitlab # [ 210.932218] gitaly[1386]: time="2026-06-17T14:54:55.469Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:25.267" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:55.467" grpc.time_ms=1.361 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868vm-test-run-nixbot-gitlab> gitlab # [ 210.952547] gitaly[1386]: time="2026-06-17T14:54:55.490Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB17Y5S8W5AYPQGPC5KHKZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:25.288" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:55.488" grpc.time_ms=1.141 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869vm-test-run-nixbot-gitlab> gitlab # [ 210.973082] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.500Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5b1c4f6f9102426f2a3dc437","created_at":"2026-06-17T14:54:54.230Z","trace_propagation_headers":{"sentry-trace":"ff992e5849c246cf91cb075205523472-3016d22981cf486d","baggage":"sentry-trace_id=ff992e5849c246cf91cb075205523472,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17Y5S8W5AYPQGPC5KHKZH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:54.235Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5b1c4f6f9102426f2a3dc437: done: 0.150337 sec","job_status":"done","queue_duration_s":1.115579,"scheduling_latency_s":1.1156,"gitaly_calls":3,"gitaly_duration_s":0.010569,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.009430000000000001,"redis_read_bytes":208,"redis_write_bytes":6802,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001398,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001467,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000315,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001677,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000479,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000221,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10417,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.122635698,"duration_s":0.150337,"completed_at":"2026-06-17T14:54:55.500Z","load_balancing_strategy":"primary","db_duration_s":0.02966,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1870vm-test-run-nixbot-gitlab> gitlab # [ 211.058045] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.502Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a6e5203ecfe58dd3cf4f6c88"],"class":"WebHooks::LogExecutionWorker","jid":"c9674b6f7c80a60292ac0c1f","created_at":"2026-06-17T14:54:55.221Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-0e0d8c74d3a84fd7","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7d8005caced473379dae6ecc83caef4689d0a6ee3816301a6f783ae0ed24e230","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:55.229Z","job_size_bytes":2507,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c9674b6f7c80a60292ac0c1f: start","job_status":"start","queue_duration_s":0.273,"scheduling_latency_s":0.273021}1871vm-test-run-nixbot-gitlab> gitlab # [ 211.085357] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.612Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a6e5203ecfe58dd3cf4f6c88"],"class":"WebHooks::LogExecutionWorker","jid":"c9674b6f7c80a60292ac0c1f","created_at":"2026-06-17T14:54:55.221Z","trace_propagation_headers":{"sentry-trace":"e7a6eb56560b483e90fd3ab4b49fb5a1-0e0d8c74d3a84fd7","baggage":"sentry-trace_id=e7a6eb56560b483e90fd3ab4b49fb5a1,sentry-release=Unknown"},"correlation_id":"01KVB17XG39NE02GMG6R8QBEJX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7d8005caced473379dae6ecc83caef4689d0a6ee3816301a6f783ae0ed24e230","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:55.229Z","job_size_bytes":2507,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c9674b6f7c80a60292ac0c1f: done: 0.110294 sec","job_status":"done","queue_duration_s":0.273,"scheduling_latency_s":0.273021,"redis_calls":10,"redis_duration_s":0.0030110000000000002,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001016,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00151,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000485,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.026,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.026,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000362057,"duration_s":0.110294,"completed_at":"2026-06-17T14:54:55.612Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1872vm-test-run-nixbot-gitlab> gitlab # [ 211.190635] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.614Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b2ff7981532de89be9cd4fd9","created_at":"2026-06-17T14:54:55.418Z","trace_propagation_headers":{"sentry-trace":"ff992e5849c246cf91cb075205523472-b694918bb4b142e6","baggage":"sentry-trace_id=ff992e5849c246cf91cb075205523472,sentry-release=Unknown"},"correlation_id":"01KVB17Y5S8W5AYPQGPC5KHKZH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:55.423Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b2ff7981532de89be9cd4fd9: start","job_status":"start","queue_duration_s":0.191503,"scheduling_latency_s":0.191525}1873vm-test-run-nixbot-gitlab> gitlab # [ 211.192439] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:55.646Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b2ff7981532de89be9cd4fd9","created_at":"2026-06-17T14:54:55.418Z","trace_propagation_headers":{"sentry-trace":"ff992e5849c246cf91cb075205523472-b694918bb4b142e6","baggage":"sentry-trace_id=ff992e5849c246cf91cb075205523472,sentry-release=Unknown"},"correlation_id":"01KVB17Y5S8W5AYPQGPC5KHKZH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:55.423Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b2ff7981532de89be9cd4fd9: done: 0.03213 sec","job_status":"done","queue_duration_s":0.191503,"scheduling_latency_s":0.191525,"redis_calls":10,"redis_duration_s":0.0022459999999999997,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000447,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001393,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355632,"duration_s":0.03213,"completed_at":"2026-06-17T14:54:55.646Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1874vm-test-run-nixbot-gitlab> gitlab # [ 211.371359] gitaly[1386]: time="2026-06-17T14:54:55.934Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB17ZR5DE3CX73KD1FVVXF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:55.925" grpc.time_ms=8.278 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1875vm-test-run-nixbot-gitlab> gitlab # [ 211.373508] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:55 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 6971876vm-test-run-nixbot-gitlab> gitlab # [ 211.432209] gitaly[1386]: time="2026-06-17T14:54:55.995Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KVB180E1WSDXM3TZPNP991S8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:55.986" pid=1386 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1877vm-test-run-nixbot-gitlab> gitlab # [ 211.433600] gitaly[1386]: time="2026-06-17T14:54:55.996Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180E1WSDXM3TZPNP991S8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:55.986" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1878vm-test-run-nixbot-gitlab> gitlab # [ 211.434814] gitaly[1386]: time="2026-06-17T14:54:55.997Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180E1WSDXM3TZPNP991S8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:55.986" grpc.time_ms=10.624 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1879vm-test-run-nixbot-gitlab> gitlab # [ 211.436261] gitlab-workhorse[1289]: time="2026-06-17T14:54:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671880vm-test-run-nixbot-gitlab> gitlab # [ 211.437111] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 601881vm-test-run-nixbot-gitlab> gitlab # [ 211.506163] gitaly[1386]: time="2026-06-17T14:54:56.068Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KVB180G1HYX6Z4K5SZR3A157 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:56.041" pid=1386 remote_ip="2001:db8:1::1" request_sha=45dab7705f24fe52a391056bf81e3c70a8a3c63b response_bytes=384 span.kind=server system=grpc1882vm-test-run-nixbot-gitlab> gitlab # [ 211.507412] gitaly[1386]: time="2026-06-17T14:54:56.069Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1210 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180G1HYX6Z4K5SZR3A157 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:56.041" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1883vm-test-run-nixbot-gitlab> gitlab # [ 211.508454] gitaly[1386]: time="2026-06-17T14:54:56.069Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1210 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180G1HYX6Z4K5SZR3A157 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-17T14:54:56.041" grpc.time_ms=27.606 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc1884vm-test-run-nixbot-gitlab> gitlab # [ 211.567652] gitlab-workhorse[1289]: time="2026-06-17T14:54:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3841885vm-test-run-nixbot-gitlab> gitlab # [ 211.568802] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 384 "" "git/2.54.0" 1281886vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=14b3955733a0a15351b056a59a904cda74dfdf9d'1887vm-test-run-nixbot-gitlab> gitlab # [ 211.607576] nixbot[3173]: {"ts": "2026-06-17T14:54:56.169Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1888vm-test-run-nixbot-gitlab> gitlab # [ 211.619411] nix-daemon[3452]: unexpected Nix daemon error: error: interrupted by the user1889vm-test-run-nixbot-gitlab> gitlab # [ 211.632520] nixbot[3173]: INFO: ::ffff:127.0.0.1:55298 - "GET /api/repos/gitlab/root/test-flake/builds?commit=14b3955733a0a15351b056a59a904cda74dfdf9d HTTP/1.1" 200 OK1890vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=14b3955733a0a15351b056a59a904cda74dfdf9d', in 0.05 seconds)1891vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-06-17T14:54:56.160888Z', 'started_at': None, 'finished_at': None}]1892vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.51 seconds)1893vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses1894vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses1895vm-test-run-nixbot-gitlab> gitlab # [ 211.663973] gitaly[1386]: time="2026-06-17T14:54:56.227Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:26.025" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:56.226" grpc.time_ms=0.587 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1896vm-test-run-nixbot-gitlab> gitlab # [ 211.705989] gitaly[1386]: time="2026-06-17T14:54:56.269Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:26.168" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:56.268" grpc.time_ms=0.536 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897vm-test-run-nixbot-gitlab> gitlab # [ 211.738274] gitaly[1386]: time="2026-06-17T14:54:56.301Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-17T14:55:26.092" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:56.292" grpc.time_ms=8.479 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898vm-test-run-nixbot-gitlab> gitlab # [ 211.762889] gitaly[1386]: time="2026-06-17T14:54:56.326Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180PVVFN1N6GCAA5MEBS2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:26.201" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:56.324" grpc.time_ms=1.791 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899vm-test-run-nixbot-gitlab> gitlab # [ 211.794677] gitaly[1386]: time="2026-06-17T14:54:56.357Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:06.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:56.350" grpc.time_ms=7.309 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900vm-test-run-nixbot-gitlab> gitlab # [ 211.798915] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 1421901vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.18 seconds)1902vm-test-run-nixbot-gitlab> []1903vm-test-run-nixbot-gitlab> gitlab # [ 211.821656] gitaly[1386]: time="2026-06-17T14:54:56.384Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=26 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=0 command.system_time_ms=25 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:06.100" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:56.350" grpc.time_ms=34.4 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904vm-test-run-nixbot-gitlab> gitlab # [ 211.857785] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:56.419Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"86dbf27f34f4bd3a77512a14","created_at":"2026-06-17T14:54:56.415Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-0ba09ff387ea476c","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.418Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-86dbf27f34f4bd3a77512a14: start","job_status":"start","queue_duration_s":0.001516,"scheduling_latency_s":0.001565}1905vm-test-run-nixbot-gitlab> gitlab # [ 211.884261] nixbot[3173]: {"ts": "2026-06-17T14:54:56.436Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/69d0fad95915071fc708273468e0ca306ba5a878 \"HTTP/1.1 201 Created\""}1906vm-test-run-nixbot-gitlab> gitlab # [ 211.885372] gitaly[1386]: time="2026-06-17T14:54:56.437Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-17T14:55:06.185" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:56.436" grpc.time_ms=0.271 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907vm-test-run-nixbot-gitlab> gitlab # [ 211.886235] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/69d0fad95915071fc708273468e0ca306ba5a878 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 2391908vm-test-run-nixbot-gitlab> gitlab # [ 211.914138] gitaly[1386]: time="2026-06-17T14:54:56.477Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-17T14:55:06.191" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:56.442" grpc.time_ms=34.799 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909vm-test-run-nixbot-gitlab> gitlab # [ 211.945246] gitaly[1386]: time="2026-06-17T14:54:56.508Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180XSCB2932VFH92YPFH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:26.406" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:56.507" grpc.time_ms=1.24 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910vm-test-run-nixbot-gitlab> gitlab # [ 211.981059] gitaly[1386]: time="2026-06-17T14:54:56.544Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:56.543" grpc.time_ms=0.432 pid=1386 span.kind=server system=grpc1911vm-test-run-nixbot-gitlab> gitlab # [ 211.983236] gitaly[1386]: time="2026-06-17T14:54:56.545Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:56.544" grpc.time_ms=0.719 pid=1386 span.kind=server system=grpc1912vm-test-run-nixbot-gitlab> gitlab # [ 211.984720] gitaly[1386]: time="2026-06-17T14:54:56.545Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB180XSCB2932VFH92YPFH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:06.197" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:56.538" grpc.time_ms=7.733 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913vm-test-run-nixbot-gitlab> gitlab # [ 212.059975] gitaly[1386]: time="2026-06-17T14:54:56.623Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:56.622" grpc.time_ms=0.455 pid=1386 span.kind=server system=grpc1914vm-test-run-nixbot-gitlab> gitlab # [ 212.061991] gitaly[1386]: time="2026-06-17T14:54:56.623Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:56.623" grpc.time_ms=0.192 pid=1386 span.kind=server system=grpc1915vm-test-run-nixbot-gitlab> gitlab # [ 212.075477] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/69d0fad95915071fc708273468e0ca306ba5a878 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1971916vm-test-run-nixbot-gitlab> gitlab # [ 212.096788] nixbot[3173]: {"ts": "2026-06-17T14:54:56.658Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/69d0fad95915071fc708273468e0ca306ba5a878 \"HTTP/1.1 201 Created\""}1917vm-test-run-nixbot-gitlab> gitlab # [ 212.110797] gitaly[1386]: time="2026-06-17T14:54:56.674Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY diskcache=77a95d84-8170-4d8a-a37a-a0d14458b54d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-17T14:55:06.249" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:56.499" pid=1386 ref=refs/keep-around/14b3955733a0a15351b056a59a904cda74dfdf9d remote_ip="2001:db8:1::1" revision=14b3955733a0a15351b056a59a904cda74dfdf9d span.kind=server system=grpc username=root1918vm-test-run-nixbot-gitlab> gitlab # [ 212.122742] gitaly[1386]: time="2026-06-17T14:54:56.686Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-17T14:54:56.685" grpc.time_ms=0.455 pid=1386 span.kind=server system=grpc1919vm-test-run-nixbot-gitlab> gitlab # [ 212.144264] gitaly[1386]: time="2026-06-17T14:54:56.707Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=148 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5000 command.oublock=8 command.real_time_ms=166 command.spawn_token_fork_ms=1 command.system_time_ms=119 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-17T14:55:06.249" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-17T14:54:56.499" grpc.time_ms=174.611 pid=1386 ref=refs/keep-around/14b3955733a0a15351b056a59a904cda74dfdf9d remote_ip="2001:db8:1::1" revision=14b3955733a0a15351b056a59a904cda74dfdf9d span.kind=server system=grpc username=root1920vm-test-run-nixbot-gitlab> gitlab # [ 212.164303] gitaly[1386]: time="2026-06-17T14:54:56.727Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:26.526" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:56.726" grpc.time_ms=0.752 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921vm-test-run-nixbot-gitlab> gitlab # [ 212.196985] gitaly[1386]: time="2026-06-17T14:54:56.760Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K diskcache=a2542c93-8711-42f2-8d85-a1a4778fd924 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-17T14:55:26.296" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:56.496" pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1922vm-test-run-nixbot-gitlab> gitlab # [ 212.213259] gitaly[1386]: time="2026-06-17T14:54:56.776Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=129 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6825 command.oublock=0 command.real_time_ms=253 command.spawn_token_fork_ms=1 command.system_time_ms=107 command.user_time_ms=22 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-17T14:55:26.296" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:54:56.496" grpc.time_ms=263.444 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923vm-test-run-nixbot-gitlab> gitlab # [ 212.283956] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/14b3955733a0a15351b056a59a904cda74dfdf9d HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 6701924vm-test-run-nixbot-gitlab> gitlab # [ 212.287619] nixbot[3173]: {"ts": "2026-06-17T14:54:56.849Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/14b3955733a0a15351b056a59a904cda74dfdf9d \"HTTP/1.1 201 Created\""}1925vm-test-run-nixbot-gitlab> gitlab # [ 212.288965] gitaly[1386]: time="2026-06-17T14:54:56.849Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-17T14:55:26.648" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:56.848" grpc.time_ms=0.804 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1926vm-test-run-nixbot-gitlab> gitlab # [ 212.295103] nixbot[3173]: {"ts": "2026-06-17T14:54:56.856Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-14b3955733a0-b2d7243a", "argv0": "bwrap"}1927vm-test-run-nixbot-gitlab> gitlab # [ 212.310552] gitaly[1386]: time="2026-06-17T14:54:56.873Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:26.672" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:56.872" grpc.time_ms=0.566 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928vm-test-run-nixbot-gitlab> gitlab # [ 212.345860] gitaly[1386]: time="2026-06-17T14:54:56.909Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:26.707" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:56.908" grpc.time_ms=0.75 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929vm-test-run-nixbot-gitlab> gitlab # [ 212.352596] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:56.915Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"10955c3da6447ee5be4fa42f","created_at":"2026-06-17T14:54:56.913Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"86dbf27f34f4bd3a77512a14","job_size_bytes":3,"pid":2150,"job_status":"deduplicated","message":"PipelineProcessWorker JID-10955c3da6447ee5be4fa42f: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1930vm-test-run-nixbot-gitlab> gitlab # [ 212.354111] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:56.917Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"86dbf27f34f4bd3a77512a14","created_at":"2026-06-17T14:54:56.415Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-0ba09ff387ea476c","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.418Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2150,"job_status":"rescheduled","message":"PipelineProcessWorker JID-86dbf27f34f4bd3a77512a14: rescheduled"}1931vm-test-run-nixbot-gitlab> gitlab # [ 212.359810] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:56.922Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"86dbf27f34f4bd3a77512a14","created_at":"2026-06-17T14:54:56.415Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-0ba09ff387ea476c","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.418Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-86dbf27f34f4bd3a77512a14: done: 0.502962 sec","job_status":"done","queue_duration_s":0.001516,"scheduling_latency_s":0.001565,"gitaly_calls":4,"gitaly_duration_s":0.29452,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.024772,"redis_read_bytes":267,"redis_write_bytes":24011,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001614,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.005825,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.003401,"redis_queues_read_bytes":27,"redis_queues_write_bytes":12390,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006591,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000288,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000667,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004386,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.153323,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.470776034,"duration_s":0.502962,"completed_at":"2026-06-17T14:54:56.922Z","load_balancing_strategy":"primary","db_duration_s":0.040243,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1932vm-test-run-nixbot-gitlab> gitlab # [ 212.428595] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:56.924Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d6b3b34d7706664dd9e88b84","created_at":"2026-06-17T14:54:56.484Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.487Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d6b3b34d7706664dd9e88b84: start","job_status":"start","queue_duration_s":0.436351,"scheduling_latency_s":0.436374}1933vm-test-run-nixbot-gitlab> gitlab # [ 212.455316] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.017Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d6b3b34d7706664dd9e88b84","created_at":"2026-06-17T14:54:56.484Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.487Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d6b3b34d7706664dd9e88b84: done: 0.093466 sec","job_status":"done","queue_duration_s":0.436351,"scheduling_latency_s":0.436374,"redis_calls":15,"redis_duration_s":0.005657,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000498,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000569,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002723,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000804,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068957,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000767974,"duration_s":0.093466,"completed_at":"2026-06-17T14:54:57.017Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01176,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1934vm-test-run-nixbot-gitlab> gitlab # [ 212.561441] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.021Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"caf88c7cbbd43b2ecbe5802e","created_at":"2026-06-17T14:54:57.000Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-fe28375fe3694490","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:419bedd1937c6190dfae7e5a0ef2b37f39ea459b0c8b6bae2bbb06e86b82b508","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:57.002Z","job_size_bytes":464,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-caf88c7cbbd43b2ecbe5802e: start","job_status":"start","queue_duration_s":0.018786,"scheduling_latency_s":0.018838}1935vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses1936vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.20 seconds)1937vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1938vm-test-run-nixbot-gitlab> gitlab # [ 212.562905] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-06-17T14:54:57.069Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"caf88c7cbbd43b2ecbe5802e","created_at":"2026-06-17T14:54:57.000Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-fe28375fe3694490","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:419bedd1937c6190dfae7e5a0ef2b37f39ea459b0c8b6bae2bbb06e86b82b508","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:57.002Z","job_size_bytes":464,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-caf88c7cbbd43b2ecbe5802e: fail: 0.047424 sec","job_status":"fail","queue_duration_s":0.018786,"scheduling_latency_s":0.018838,"redis_calls":7,"redis_duration_s":0.002372,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001751,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000621,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047424,"completed_at":"2026-06-17T14:54:57.068Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.005619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1939vm-test-run-nixbot-gitlab> gitlab # [ 213.160605] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.071Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9e69b7e097321079e083d79f","created_at":"2026-06-17T14:54:56.733Z","trace_propagation_headers":{"sentry-trace":"7f4e0401d7e1456db211dde6df51bfd2-bfc463e3d5294d23","baggage":"sentry-trace_id=7f4e0401d7e1456db211dde6df51bfd2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB180NGYEFC9X1M42A4GKKY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.738Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9e69b7e097321079e083d79f: start","job_status":"start","queue_duration_s":0.333175,"scheduling_latency_s":0.333207}1940vm-test-run-nixbot-gitlab> gitlab # [ 213.186974] gitaly[1386]: time="2026-06-17T14:54:57.467Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=10 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=10 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-17T14:55:27.254" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:57.455" grpc.time_ms=11.746 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1941vm-test-run-nixbot-gitlab> gitlab # [ 213.207652] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 611942vm-test-run-nixbot-gitlab> gitlab # [ 213.211255] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.214Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9e69b7e097321079e083d79f","created_at":"2026-06-17T14:54:56.733Z","trace_propagation_headers":{"sentry-trace":"7f4e0401d7e1456db211dde6df51bfd2-bfc463e3d5294d23","baggage":"sentry-trace_id=7f4e0401d7e1456db211dde6df51bfd2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB180NGYEFC9X1M42A4GKKY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.738Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9e69b7e097321079e083d79f: done: 0.14276 sec","job_status":"done","queue_duration_s":0.333175,"scheduling_latency_s":0.333207,"redis_calls":7,"redis_duration_s":0.002288,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001675,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000613,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.12,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.119,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.14276,"completed_at":"2026-06-17T14:54:57.214Z","load_balancing_strategy":"primary","db_duration_s":0.119677,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1943vm-test-run-nixbot-gitlab> gitlab # [ 213.280376] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.215Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"31a57bd542690e9fe3d663c8","created_at":"2026-06-17T14:54:56.739Z","trace_propagation_headers":{"sentry-trace":"7f4e0401d7e1456db211dde6df51bfd2-bfc463e3d5294d23","baggage":"sentry-trace_id=7f4e0401d7e1456db211dde6df51bfd2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB180NGYEFC9X1M42A4GKKY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.741Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-31a57bd542690e9fe3d663c8: start","job_status":"start","queue_duration_s":0.473995,"scheduling_latency_s":0.474026}1944vm-test-run-nixbot-gitlab> gitlab # [ 213.302126] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.254Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"31a57bd542690e9fe3d663c8","created_at":"2026-06-17T14:54:56.739Z","trace_propagation_headers":{"sentry-trace":"7f4e0401d7e1456db211dde6df51bfd2-bfc463e3d5294d23","baggage":"sentry-trace_id=7f4e0401d7e1456db211dde6df51bfd2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB180NGYEFC9X1M42A4GKKY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.741Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-31a57bd542690e9fe3d663c8: done: 0.03874 sec","job_status":"done","queue_duration_s":0.473995,"scheduling_latency_s":0.474026,"redis_calls":10,"redis_duration_s":0.00284,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03874,"completed_at":"2026-06-17T14:54:57.254Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1945vm-test-run-nixbot-gitlab> gitlab # [ 213.420468] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.255Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"35751f4374f4e387e613ceeb","created_at":"2026-06-17T14:54:56.783Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.788Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-35751f4374f4e387e613ceeb: start","job_status":"start","queue_duration_s":0.467739,"scheduling_latency_s":0.467763}1946vm-test-run-nixbot-gitlab> gitlab # [ 213.421912] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.284Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"35751f4374f4e387e613ceeb","created_at":"2026-06-17T14:54:56.783Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.788Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-35751f4374f4e387e613ceeb: done: 0.028484 sec","job_status":"done","queue_duration_s":0.467739,"scheduling_latency_s":0.467763,"redis_calls":10,"redis_duration_s":0.004593,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000869105,"duration_s":0.028484,"completed_at":"2026-06-17T14:54:57.284Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005834,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1947vm-test-run-nixbot-gitlab> gitlab # [ 213.465813] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.286Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"57117c27281d6e5dd1de5083","created_at":"2026-06-17T14:54:56.801Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.807Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-57117c27281d6e5dd1de5083: start","job_status":"start","queue_duration_s":0.478984,"scheduling_latency_s":0.479006}1948vm-test-run-nixbot-gitlab> gitlab # [ 213.491106] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.313Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"57117c27281d6e5dd1de5083","created_at":"2026-06-17T14:54:56.801Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.807Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-57117c27281d6e5dd1de5083: done: 0.027141 sec","job_status":"done","queue_duration_s":0.478984,"scheduling_latency_s":0.479006,"redis_calls":10,"redis_duration_s":0.002257,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000467,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001353,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000393905,"duration_s":0.027141,"completed_at":"2026-06-17T14:54:57.313Z","load_balancing_strategy":"primary","db_duration_s":0.004755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1949vm-test-run-nixbot-gitlab> gitlab # [ 213.564347] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.314Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a2c3ff7053292b576d764893","created_at":"2026-06-17T14:54:56.817Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.820Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a2c3ff7053292b576d764893: start","job_status":"start","queue_duration_s":0.493554,"scheduling_latency_s":0.493575}1950vm-test-run-nixbot-gitlab> gitlab # [ 213.583372] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.340Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a2c3ff7053292b576d764893","created_at":"2026-06-17T14:54:56.817Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.820Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a2c3ff7053292b576d764893: done: 0.026244 sec","job_status":"done","queue_duration_s":0.493554,"scheduling_latency_s":0.493575,"redis_calls":13,"redis_duration_s":0.002938,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000686,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346972,"duration_s":0.026244,"completed_at":"2026-06-17T14:54:57.340Z","load_balancing_strategy":"primary","db_duration_s":0.003042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1951vm-test-run-nixbot-gitlab> gitlab # [ 213.707359] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.342Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"014ead78ca6fc3a4ace08b27","created_at":"2026-06-17T14:54:56.821Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.826Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-014ead78ca6fc3a4ace08b27: start","job_status":"start","queue_duration_s":0.515184,"scheduling_latency_s":0.515204}1952vm-test-run-nixbot-gitlab> gitlab # [ 213.708876] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.364Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"014ead78ca6fc3a4ace08b27","created_at":"2026-06-17T14:54:56.821Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.826Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-014ead78ca6fc3a4ace08b27: done: 0.022058 sec","job_status":"done","queue_duration_s":0.515184,"scheduling_latency_s":0.515204,"redis_calls":10,"redis_duration_s":0.0022630000000000003,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345574,"duration_s":0.022058,"completed_at":"2026-06-17T14:54:57.364Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953vm-test-run-nixbot-gitlab> gitlab # [ 213.751775] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.365Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"b03a2bad8a6fb166a637b271","created_at":"2026-06-17T14:54:56.827Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.832Z","job_size_bytes":19,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-b03a2bad8a6fb166a637b271: start","job_status":"start","queue_duration_s":0.532428,"scheduling_latency_s":0.532495}1954vm-test-run-nixbot-gitlab> gitlab # [ 213.777923] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.389Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"b03a2bad8a6fb166a637b271","created_at":"2026-06-17T14:54:56.827Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.832Z","job_size_bytes":19,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-b03a2bad8a6fb166a637b271: done: 0.024396 sec","job_status":"done","queue_duration_s":0.532428,"scheduling_latency_s":0.532495,"redis_calls":11,"redis_duration_s":0.00327,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002263,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000589,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039586,"duration_s":0.024396,"completed_at":"2026-06-17T14:54:57.389Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002791,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1955vm-test-run-nixbot-gitlab> gitlab # [ 213.852203] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.391Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"27e3028ac2f59a5ca0f38d27","created_at":"2026-06-17T14:54:56.829Z","trace_propagation_headers":{"sentry-trace":"7f4e0401d7e1456db211dde6df51bfd2-bfc463e3d5294d23","baggage":"sentry-trace_id=7f4e0401d7e1456db211dde6df51bfd2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB180NGYEFC9X1M42A4GKKY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.834Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-27e3028ac2f59a5ca0f38d27: start","job_status":"start","queue_duration_s":0.557195,"scheduling_latency_s":0.557217}1956vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses1957vm-test-run-nixbot-gitlab> gitlab # [ 213.938350] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.531Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"27e3028ac2f59a5ca0f38d27","created_at":"2026-06-17T14:54:56.829Z","trace_propagation_headers":{"sentry-trace":"7f4e0401d7e1456db211dde6df51bfd2-bfc463e3d5294d23","baggage":"sentry-trace_id=7f4e0401d7e1456db211dde6df51bfd2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB180NGYEFC9X1M42A4GKKY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.834Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-27e3028ac2f59a5ca0f38d27: done: 0.140227 sec","job_status":"done","queue_duration_s":0.557195,"scheduling_latency_s":0.557217,"gitaly_calls":3,"gitaly_duration_s":0.029345,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008959,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001022,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002168,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000317,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002696,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000248,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000446,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077304,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.119585869,"duration_s":0.140227,"completed_at":"2026-06-17T14:54:57.531Z","load_balancing_strategy":"primary","db_duration_s":0.02844,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1958vm-test-run-nixbot-gitlab> gitlab # [ 213.953858] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.533Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a532b52984be2ebff423a63f","created_at":"2026-06-17T14:54:56.835Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.840Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a532b52984be2ebff423a63f: start","job_status":"start","queue_duration_s":0.692474,"scheduling_latency_s":0.692497}1959vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.10 seconds)1960vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1961vm-test-run-nixbot-gitlab> gitlab # [ 213.981522] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.556Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a532b52984be2ebff423a63f","created_at":"2026-06-17T14:54:56.835Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.840Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a532b52984be2ebff423a63f: done: 0.023946 sec","job_status":"done","queue_duration_s":0.692474,"scheduling_latency_s":0.692497,"redis_calls":10,"redis_duration_s":0.002628,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000655111,"duration_s":0.023946,"completed_at":"2026-06-17T14:54:57.556Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002977,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962vm-test-run-nixbot-gitlab> gitlab # [ 214.109700] gitaly[1386]: time="2026-06-17T14:54:57.498Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:27.297" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:57.497" grpc.time_ms=0.805 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963vm-test-run-nixbot-gitlab> gitlab # [ 214.110880] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.559Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"3e6fe85b16aac5d1932cc145","created_at":"2026-06-17T14:54:56.841Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.842Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3e6fe85b16aac5d1932cc145: start","job_status":"start","queue_duration_s":0.716097,"scheduling_latency_s":0.71612}1964vm-test-run-nixbot-gitlab> gitlab # [ 214.111781] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.584Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"3e6fe85b16aac5d1932cc145","created_at":"2026-06-17T14:54:56.841Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.842Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3e6fe85b16aac5d1932cc145: done: 0.02509 sec","job_status":"done","queue_duration_s":0.716097,"scheduling_latency_s":0.71612,"redis_calls":7,"redis_duration_s":0.002835,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002353,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02509,"completed_at":"2026-06-17T14:54:57.584Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003211,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965vm-test-run-nixbot-gitlab> gitlab # [ 214.172565] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.585Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2dc3b9247a57ffbc6a8e612a","created_at":"2026-06-17T14:54:56.874Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.880Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2dc3b9247a57ffbc6a8e612a: start","job_status":"start","queue_duration_s":0.705241,"scheduling_latency_s":0.705264}1966vm-test-run-nixbot-gitlab> gitlab # [ 214.198328] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.613Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2dc3b9247a57ffbc6a8e612a","created_at":"2026-06-17T14:54:56.874Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.880Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2dc3b9247a57ffbc6a8e612a: done: 0.027606 sec","job_status":"done","queue_duration_s":0.705241,"scheduling_latency_s":0.705264,"redis_calls":10,"redis_duration_s":0.002298,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001389,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000354514,"duration_s":0.027606,"completed_at":"2026-06-17T14:54:57.613Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007293,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1967vm-test-run-nixbot-gitlab> gitlab # [ 214.293301] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.614Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7810bd3a18eef9910c8b7061","created_at":"2026-06-17T14:54:56.881Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.884Z","job_size_bytes":6,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7810bd3a18eef9910c8b7061: start","job_status":"start","queue_duration_s":0.729786,"scheduling_latency_s":0.729806}1968vm-test-run-nixbot-gitlab> gitlab # [ 214.294535] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.633Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7810bd3a18eef9910c8b7061","created_at":"2026-06-17T14:54:56.881Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.884Z","job_size_bytes":6,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7810bd3a18eef9910c8b7061: done: 0.018836 sec","job_status":"done","queue_duration_s":0.729786,"scheduling_latency_s":0.729806,"redis_calls":10,"redis_duration_s":0.002275,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000348368,"duration_s":0.018836,"completed_at":"2026-06-17T14:54:57.633Z","load_balancing_strategy":"primary","db_duration_s":0.001649,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1969vm-test-run-nixbot-gitlab> gitlab # [ 214.365098] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.634Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"5ef35e42745e7c56c6b1e351","created_at":"2026-06-17T14:54:56.917Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.918Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5ef35e42745e7c56c6b1e351: start","job_status":"start","queue_duration_s":0.716182,"scheduling_latency_s":0.716203}1970vm-test-run-nixbot-gitlab> gitlab # [ 214.438141] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.702Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"5ef35e42745e7c56c6b1e351","created_at":"2026-06-17T14:54:56.917Z","trace_propagation_headers":{"sentry-trace":"5c60e5c52f05415588b91e94b438c7cf-b93e2e7d67154694","baggage":"sentry-trace_id=5c60e5c52f05415588b91e94b438c7cf,sentry-release=Unknown"},"correlation_id":"01KVB180P33K7DJ44BW22FHW3K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:56.918Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5ef35e42745e7c56c6b1e351: done: 0.067808 sec","job_status":"done","queue_duration_s":0.716182,"scheduling_latency_s":0.716203,"gitaly_calls":2,"gitaly_duration_s":0.006634,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003491,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000675,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0016,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000222,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.045447397,"duration_s":0.067808,"completed_at":"2026-06-17T14:54:57.702Z","load_balancing_strategy":"primary","db_duration_s":0.01244,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1971vm-test-run-nixbot-gitlab> gitlab # [ 214.463107] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.704Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b2e8ce642fccbe95c9494e79","created_at":"2026-06-17T14:54:57.436Z","trace_propagation_headers":{"sentry-trace":"7f4e0401d7e1456db211dde6df51bfd2-62e1e228e44e4c60","baggage":"sentry-trace_id=7f4e0401d7e1456db211dde6df51bfd2,sentry-release=Unknown"},"correlation_id":"01KVB180NGYEFC9X1M42A4GKKY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:57.438Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b2e8ce642fccbe95c9494e79: start","job_status":"start","queue_duration_s":0.265976,"scheduling_latency_s":0.266005}1972vm-test-run-nixbot-gitlab> gitlab # [ 214.489384] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:54:57.729Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b2e8ce642fccbe95c9494e79","created_at":"2026-06-17T14:54:57.436Z","trace_propagation_headers":{"sentry-trace":"7f4e0401d7e1456db211dde6df51bfd2-62e1e228e44e4c60","baggage":"sentry-trace_id=7f4e0401d7e1456db211dde6df51bfd2,sentry-release=Unknown"},"correlation_id":"01KVB180NGYEFC9X1M42A4GKKY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-17T14:54:57.438Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b2e8ce642fccbe95c9494e79: done: 0.024867 sec","job_status":"done","queue_duration_s":0.265976,"scheduling_latency_s":0.266005,"redis_calls":7,"redis_duration_s":0.001627,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024867,"completed_at":"2026-06-17T14:54:57.729Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973vm-test-run-nixbot-gitlab> gitlab # [ 214.585109] gitaly[1386]: time="2026-06-17T14:54:57.522Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180NGYEFC9X1M42A4GKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:27.320" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:57.521" grpc.time_ms=0.653 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974vm-test-run-nixbot-gitlab> gitlab # [ 214.586685] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 541975vm-test-run-nixbot-gitlab> gitlab # [ 214.587756] gitaly[1386]: time="2026-06-17T14:54:57.557Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB181ZEA5FZRM0EX701F631 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:27.356" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:57.556" grpc.time_ms=0.642 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976vm-test-run-nixbot-gitlab> gitlab # [ 214.601441] gitaly[1386]: time="2026-06-17T14:54:57.666Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-17T14:55:27.464" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:57.665" grpc.time_ms=1.017 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977vm-test-run-nixbot-gitlab> gitlab # [ 214.689293] gitaly[1386]: time="2026-06-17T14:54:57.690Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB180P33K7DJ44BW22FHW3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:27.489" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:57.689" grpc.time_ms=0.634 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1978vm-test-run-nixbot-gitlab> gitlab # [ 214.690562] gitaly[1386]: time="2026-06-17T14:54:58.663Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB18322JR4B6CPCZQBFCFM8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:28.461" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:58.662" grpc.time_ms=0.826 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979vm-test-run-nixbot-gitlab> gitlab # [ 214.787875] nix-daemon[3447]: accepted connection from pid 3589, user nixbot1980vm-test-run-nixbot-gitlab> gitlab # [ 214.871357] nix-daemon[3447]: accepted connection from pid 3588, user nixbot1981vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses1982vm-test-run-nixbot-gitlab> gitlab # [ 215.247539] gitaly[1386]: time="2026-06-17T14:54:59.810Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB1844TDTZSG1CXQH80Z0Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:29.207" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:54:59.808" grpc.time_ms=1.431 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983vm-test-run-nixbot-gitlab> gitlab # [ 215.301094] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:54:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1251984vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.19 seconds)1985vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1986vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses1987vm-test-run-nixbot-gitlab> gitlab # [ 216.275145] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:00.836Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f27610d6dc04905766af11c3","created_at":"2026-06-17T14:54:44.443Z","trace_propagation_headers":{"sentry-trace":"669503caec8549e999b1c6c8cfaf272b-61ef91da62ed423d","baggage":"sentry-trace_id=669503caec8549e999b1c6c8cfaf272b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17MYH4QZEM494QTGEK8XQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9c434829509957b6e55b619bbe14c1930e4cbadc89fd07f4f905130b68c42ce1","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:00.832Z","retry_count":0,"job_size_bytes":451,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f27610d6dc04905766af11c3: start","job_status":"start","queue_duration_s":0.004006,"scheduling_latency_s":0.004081}1988vm-test-run-nixbot-gitlab> gitlab # [ 216.347781] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-06-17T14:55:00.910Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f27610d6dc04905766af11c3","created_at":"2026-06-17T14:54:44.443Z","trace_propagation_headers":{"sentry-trace":"669503caec8549e999b1c6c8cfaf272b-61ef91da62ed423d","baggage":"sentry-trace_id=669503caec8549e999b1c6c8cfaf272b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB17MYH4QZEM494QTGEK8XQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9c434829509957b6e55b619bbe14c1930e4cbadc89fd07f4f905130b68c42ce1","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:00.832Z","retry_count":0,"job_size_bytes":451,"pid":2150,"sidekiq_tid":"m5jq","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f27610d6dc04905766af11c3: fail: 0.072508 sec","job_status":"fail","queue_duration_s":0.004006,"scheduling_latency_s":0.004081,"redis_calls":7,"redis_duration_s":0.004526,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003194,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001332,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.072508,"completed_at":"2026-06-17T14:55:00.908Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.015833,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1989vm-test-run-nixbot-gitlab> gitlab # [ 217.007675] gitaly[1386]: time="2026-06-17T14:55:01.570Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KVB185WKDXK27SWDKQ069GV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:31.364" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:01.565" grpc.time_ms=4.619 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990vm-test-run-nixbot-gitlab> gitlab # [ 217.014330] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:01.577Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"fbcf6582741ea57c224c4057","created_at":"2026-06-17T14:55:01.565Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76f9a57a12966164eeeb27ae85bf5a78","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.574Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-fbcf6582741ea57c224c4057: start","job_status":"start","queue_duration_s":0.002261,"scheduling_latency_s":0.002303}1991vm-test-run-nixbot-gitlab> gitlab # [ 217.046685] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:01.609Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1992vm-test-run-nixbot-gitlab> gitlab # [ 217.081476] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1211993vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.78 seconds)1994vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1995vm-test-run-nixbot-gitlab> gitlab # [ 217.086794] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:01.648Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"fbcf6582741ea57c224c4057","created_at":"2026-06-17T14:55:01.565Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76f9a57a12966164eeeb27ae85bf5a78","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.574Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-fbcf6582741ea57c224c4057: done: 0.071769 sec","job_status":"done","queue_duration_s":0.002261,"scheduling_latency_s":0.002303,"redis_calls":7,"redis_duration_s":0.008425,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003976,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004449,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.071769,"completed_at":"2026-06-17T14:55:01.648Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1996vm-test-run-nixbot-gitlab> gitlab # [ 217.123195] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:01.655Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"6020ed78635244e0f6b274c9","created_at":"2026-06-17T14:55:01.584Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.602Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-6020ed78635244e0f6b274c9: start","job_status":"start","queue_duration_s":0.053464,"scheduling_latency_s":0.053498}1997vm-test-run-nixbot-gitlab> gitlab # [ 217.307326] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:01.868Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"6020ed78635244e0f6b274c9","created_at":"2026-06-17T14:55:01.584Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.602Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-6020ed78635244e0f6b274c9: done: 0.212977 sec","job_status":"done","queue_duration_s":0.053464,"scheduling_latency_s":0.053498,"redis_calls":28,"redis_duration_s":0.12432499999999999,"redis_read_bytes":36,"redis_write_bytes":10593,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.010588,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.001326,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8305,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.051994,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.060417,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.027959699,"duration_s":0.212977,"completed_at":"2026-06-17T14:55:01.868Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1998vm-test-run-nixbot-gitlab> gitlab # [ 217.417858] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:01.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8e4b03dcc47d81f50dc9495b","created_at":"2026-06-17T14:55:01.632Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e890d88b98735250d776eff4df374f1f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-17T14:55:01.638Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8e4b03dcc47d81f50dc9495b: start","job_status":"start","queue_duration_s":0.342061,"scheduling_latency_s":0.342256}1999vm-test-run-nixbot-gitlab> gitlab # [ 217.466553] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:02.029Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2000vm-test-run-nixbot-gitlab> gitlab # [ 217.530150] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:02.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8e4b03dcc47d81f50dc9495b","created_at":"2026-06-17T14:55:01.632Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e890d88b98735250d776eff4df374f1f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-17T14:55:01.638Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8e4b03dcc47d81f50dc9495b: done: 0.112138 sec","job_status":"done","queue_duration_s":0.342061,"scheduling_latency_s":0.342256,"redis_calls":7,"redis_duration_s":0.033055,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.02838,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004675,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.112138,"completed_at":"2026-06-17T14:55:02.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2001vm-test-run-nixbot-gitlab> gitlab # [ 217.619809] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:02.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"45082040aa08d1169a6f0403","created_at":"2026-06-17T14:55:01.650Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ef19f3ad32d5829ac28a375a4ee2baa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-17T14:55:01.655Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-45082040aa08d1169a6f0403: start","job_status":"start","queue_duration_s":0.526969,"scheduling_latency_s":0.527037}2002vm-test-run-nixbot-gitlab> gitlab # [ 217.673331] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:02.236Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2003vm-test-run-nixbot-gitlab> gitlab # [ 217.726289] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:02.288Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"45082040aa08d1169a6f0403","created_at":"2026-06-17T14:55:01.650Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ef19f3ad32d5829ac28a375a4ee2baa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-17T14:55:01.655Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-45082040aa08d1169a6f0403: done: 0.106764 sec","job_status":"done","queue_duration_s":0.526969,"scheduling_latency_s":0.527037,"redis_calls":7,"redis_duration_s":0.025633999999999997,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.017608,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.008026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.106764,"completed_at":"2026-06-17T14:55:02.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004vm-test-run-nixbot-gitlab> gitlab # [ 217.811133] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:02.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"51977a05d92bb91c24698d69","created_at":"2026-06-17T14:55:01.667Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69294614710304c3be8b87f0bf168253","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.673Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-51977a05d92bb91c24698d69: start","job_status":"start","queue_duration_s":0.700468,"scheduling_latency_s":0.700536}2005vm-test-run-nixbot-gitlab> gitlab # [ 217.929218] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:02.492Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2006vm-test-run-nixbot-gitlab> gitlab # [ 217.986778] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:02.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"51977a05d92bb91c24698d69","created_at":"2026-06-17T14:55:01.667Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69294614710304c3be8b87f0bf168253","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.673Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-51977a05d92bb91c24698d69: done: 0.175342 sec","job_status":"done","queue_duration_s":0.700468,"scheduling_latency_s":0.700536,"redis_calls":7,"redis_duration_s":0.028683,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.021406,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007277,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.175342,"completed_at":"2026-06-17T14:55:02.548Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2007vm-test-run-nixbot-gitlab> gitlab # [ 217.996900] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:02.559Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0f593bf84feedaa2166db277","created_at":"2026-06-17T14:55:01.682Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.685Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0f593bf84feedaa2166db277: start","job_status":"start","queue_duration_s":0.873628,"scheduling_latency_s":0.873694}2008vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses2009vm-test-run-nixbot-gitlab> gitlab # [ 218.181277] gitaly[1386]: time="2026-06-17T14:55:02.742Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB18719TX9GK0A7FBC5RFS9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:32.540" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:02.740" grpc.time_ms=0.842 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2010vm-test-run-nixbot-gitlab> gitlab # [ 218.196306] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 592011vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.12 seconds)2012vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2013vm-test-run-nixbot-gitlab> gitlab # [ 218.714966] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:03.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0f593bf84feedaa2166db277","created_at":"2026-06-17T14:55:01.682Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.685Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0f593bf84feedaa2166db277: done: 0.71628 sec","job_status":"done","queue_duration_s":0.873628,"scheduling_latency_s":0.873694,"redis_calls":60,"redis_duration_s":0.178253,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.003313,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.168051,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006889,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.246295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.71628,"completed_at":"2026-06-17T14:55:03.275Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014vm-test-run-nixbot-gitlab> gitlab # [ 218.806293] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:03.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f4c69d753e8b699ccb651948","created_at":"2026-06-17T14:55:01.753Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"800ed579e1d3d3e9a035a939febc7334","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.767Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f4c69d753e8b699ccb651948: start","job_status":"start","queue_duration_s":1.600526,"scheduling_latency_s":1.600605}2015vm-test-run-nixbot-gitlab> gitlab # [ 218.931210] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:03.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f4c69d753e8b699ccb651948","created_at":"2026-06-17T14:55:01.753Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"800ed579e1d3d3e9a035a939febc7334","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.767Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f4c69d753e8b699ccb651948: done: 0.125061 sec","job_status":"done","queue_duration_s":1.600526,"scheduling_latency_s":1.600605,"redis_calls":13,"redis_duration_s":0.032133,"redis_read_bytes":11,"redis_write_bytes":1803,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.009148,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":716,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00855,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.003827022,"exclusive_lock_hold_duration_s":0.004940851,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.125061,"completed_at":"2026-06-17T14:55:03.493Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008911,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2016vm-test-run-nixbot-gitlab> gitlab # [ 218.943159] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:03.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"fe6fa1feb035fed37c5115c1","created_at":"2026-06-17T14:55:01.792Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cadd8e585bfcb76e01a10299d126a051","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.799Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-fe6fa1feb035fed37c5115c1: start","job_status":"start","queue_duration_s":1.706293,"scheduling_latency_s":1.706362}2017vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses2018vm-test-run-nixbot-gitlab> gitlab # [ 219.320282] gitaly[1386]: time="2026-06-17T14:55:03.880Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB1884DMVAP9YR2GAMF5Z2Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:33.679" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:03.879" grpc.time_ms=0.747 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2019vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.13 seconds)2020vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2021vm-test-run-nixbot-gitlab> gitlab # [ 219.340897] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 772022vm-test-run-nixbot-gitlab> gitlab # [ 219.383525] nix-daemon[3447]: accepted connection from pid 3624, user nixbot2023vm-test-run-nixbot-gitlab> gitlab # [ 219.409425] gitaly[1386]: time="2026-06-17T14:55:03.972Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB1887W5CZ758A9CBF71QN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:33.770" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:03.971" grpc.time_ms=0.666 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2024vm-test-run-nixbot-gitlab> gitlab # [ 219.438421] gitaly[1386]: time="2026-06-17T14:55:04.001Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KVB1887W5CZ758A9CBF71QN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:13.193" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:55:03.994" grpc.time_ms=7.437 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025vm-test-run-nixbot-gitlab> gitlab # [ 219.498850] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/14b3955733a0a15351b056a59a904cda74dfdf9d HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 1292026vm-test-run-nixbot-gitlab> gitlab # [ 219.501297] nixbot[3173]: {"ts": "2026-06-17T14:55:04.064Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/14b3955733a0a15351b056a59a904cda74dfdf9d \"HTTP/1.1 201 Created\""}2027vm-test-run-nixbot-gitlab> gitlab # [ 219.550390] gitaly[1386]: time="2026-06-17T14:55:04.113Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB188C7CC36C9M3X0CJDWQZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:34.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:04.112" grpc.time_ms=0.949 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2028vm-test-run-nixbot-gitlab> gitlab # [ 219.586534] gitaly[1386]: time="2026-06-17T14:55:04.149Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB188C7CC36C9M3X0CJDWQZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:14.021" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:55:04.142" grpc.time_ms=6.553 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2029vm-test-run-nixbot-gitlab> gitlab # [ 219.612791] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.174Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"fe6fa1feb035fed37c5115c1","created_at":"2026-06-17T14:55:01.792Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cadd8e585bfcb76e01a10299d126a051","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.799Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-fe6fa1feb035fed37c5115c1: done: 0.669048 sec","job_status":"done","queue_duration_s":1.706293,"scheduling_latency_s":1.706362,"redis_calls":11,"redis_duration_s":0.018763,"redis_read_bytes":9,"redis_write_bytes":1346,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004549,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007728,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.187,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.074,"db_ci_duration_s":0.112,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.212566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002777169,"exclusive_lock_hold_duration_s":0.533846061,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.669048,"completed_at":"2026-06-17T14:55:04.174Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.428281,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030vm-test-run-nixbot-gitlab> gitlab # [ 219.626355] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/14b3955733a0a15351b056a59a904cda74dfdf9d HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1172031vm-test-run-nixbot-gitlab> gitlab # [ 219.627801] nixbot[3173]: {"ts": "2026-06-17T14:55:04.190Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/14b3955733a0a15351b056a59a904cda74dfdf9d \"HTTP/1.1 201 Created\""}2032vm-test-run-nixbot-gitlab> gitlab # [ 219.696295] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.259Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"f401e0826a08d59a2e2f2051","created_at":"2026-06-17T14:55:01.812Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.820Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-f401e0826a08d59a2e2f2051: start","job_status":"start","queue_duration_s":2.439109,"scheduling_latency_s":2.439155}2033vm-test-run-nixbot-gitlab> gitlab # [ 219.841919] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"f401e0826a08d59a2e2f2051","created_at":"2026-06-17T14:55:01.812Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.820Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-f401e0826a08d59a2e2f2051: done: 0.144036 sec","job_status":"done","queue_duration_s":2.439109,"scheduling_latency_s":2.439155,"redis_calls":16,"redis_duration_s":0.036987,"redis_read_bytes":18,"redis_write_bytes":4401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004059,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":371,"redis_queues_calls":2,"redis_queues_duration_s":0.003179,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.022733,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.007016,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.144036,"completed_at":"2026-06-17T14:55:04.403Z","load_balancing_strategy":"primary","db_duration_s":0.00623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2034vm-test-run-nixbot-gitlab> gitlab # [ 219.854279] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8277e9d373f96ec63c731e97","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8277e9d373f96ec63c731e97: start","job_status":"start","queue_duration_s":2.595881,"scheduling_latency_s":2.595943}2035vm-test-run-nixbot-gitlab> gitlab # [ 219.959529] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8277e9d373f96ec63c731e97","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8277e9d373f96ec63c731e97: done: 0.10523 sec","job_status":"done","queue_duration_s":2.595881,"scheduling_latency_s":2.595943,"redis_calls":16,"redis_duration_s":0.028214,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006712,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014711,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00084033,"duration_s":0.10523,"completed_at":"2026-06-17T14:55:04.522Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008751,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2036vm-test-run-nixbot-gitlab> gitlab # [ 220.035239] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"fa7c6a85bbe858005c32c5ee","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-fa7c6a85bbe858005c32c5ee: start","job_status":"start","queue_duration_s":2.776638,"scheduling_latency_s":2.776775}2037vm-test-run-nixbot-gitlab> gitlab # [ 220.124425] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"fa7c6a85bbe858005c32c5ee","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-fa7c6a85bbe858005c32c5ee: done: 0.089169 sec","job_status":"done","queue_duration_s":2.776638,"scheduling_latency_s":2.776775,"redis_calls":13,"redis_duration_s":0.033767,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009799,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.023968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.089169,"completed_at":"2026-06-17T14:55:04.686Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2038vm-test-run-nixbot-gitlab> gitlab # [ 220.213409] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e8e9260ac1f63260b5bdebb1","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e8e9260ac1f63260b5bdebb1: start","job_status":"start","queue_duration_s":2.95447,"scheduling_latency_s":2.954512}2039vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses2040vm-test-run-nixbot-gitlab> gitlab # [ 220.327268] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e8e9260ac1f63260b5bdebb1","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e8e9260ac1f63260b5bdebb1: done: 0.114226 sec","job_status":"done","queue_duration_s":2.95447,"scheduling_latency_s":2.954512,"redis_calls":13,"redis_duration_s":0.029264000000000002,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009801,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.019463,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.114226,"completed_at":"2026-06-17T14:55:04.889Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005176,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2041vm-test-run-nixbot-gitlab> gitlab # [ 220.425123] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:04.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9e805651dfada9c8edbc5553","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9e805651dfada9c8edbc5553: start","job_status":"start","queue_duration_s":3.16502,"scheduling_latency_s":3.16506}2042vm-test-run-nixbot-gitlab> gitlab # [ 220.458540] gitaly[1386]: time="2026-06-17T14:55:05.020Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB1897QDPFT3MCT09QW6K1Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:35.018" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:05.019" grpc.time_ms=1.162 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2043vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.16 seconds)2044vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': '2026-06-17T14:55:04.038Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-17T14:55:04.169Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2045vm-test-run-nixbot-gitlab> gitlab # [ 220.496377] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1072046vm-test-run-nixbot-gitlab> gitlab # [ 220.501298] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9e805651dfada9c8edbc5553","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9e805651dfada9c8edbc5553: done: 0.07653 sec","job_status":"done","queue_duration_s":3.16502,"scheduling_latency_s":3.16506,"redis_calls":13,"redis_duration_s":0.010336,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001781,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008555,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.07653,"completed_at":"2026-06-17T14:55:05.062Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017813,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047vm-test-run-nixbot-gitlab> gitlab # [ 220.568109] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e80025764be89c03839d0fc0","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e80025764be89c03839d0fc0: start","job_status":"start","queue_duration_s":3.309808,"scheduling_latency_s":3.309858}2048vm-test-run-nixbot-gitlab> gitlab # [ 220.665173] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e80025764be89c03839d0fc0","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e80025764be89c03839d0fc0: done: 0.096594 sec","job_status":"done","queue_duration_s":3.309808,"scheduling_latency_s":3.309858,"redis_calls":13,"redis_duration_s":0.028911,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.096594,"completed_at":"2026-06-17T14:55:05.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003012,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049vm-test-run-nixbot-gitlab> gitlab # [ 220.746169] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"41d0e14955a60909b625200e","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-41d0e14955a60909b625200e: start","job_status":"start","queue_duration_s":3.415599,"scheduling_latency_s":3.415657}2050vm-test-run-nixbot-gitlab> gitlab # [ 220.767552] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"41d0e14955a60909b625200e","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-41d0e14955a60909b625200e: done: 0.070463 sec","job_status":"done","queue_duration_s":3.415599,"scheduling_latency_s":3.415657,"redis_calls":13,"redis_duration_s":0.025587000000000002,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.018089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.070463,"completed_at":"2026-06-17T14:55:05.307Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2051vm-test-run-nixbot-gitlab> gitlab # [ 220.769311] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f38bbe56773e9b4f8706634d","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f38bbe56773e9b4f8706634d: start","job_status":"start","queue_duration_s":3.495541,"scheduling_latency_s":3.495597}2052vm-test-run-nixbot-gitlab> gitlab # [ 220.824717] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f38bbe56773e9b4f8706634d","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f38bbe56773e9b4f8706634d: done: 0.070623 sec","job_status":"done","queue_duration_s":3.495541,"scheduling_latency_s":3.495597,"redis_calls":13,"redis_duration_s":0.021269,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004023,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.017246,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.070623,"completed_at":"2026-06-17T14:55:05.387Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053vm-test-run-nixbot-gitlab> gitlab # [ 220.845233] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"15eb5ad188cb6200cf349068","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-15eb5ad188cb6200cf349068: start","job_status":"start","queue_duration_s":3.571597,"scheduling_latency_s":3.571635}2054vm-test-run-nixbot-gitlab> gitlab # [ 220.879253] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"15eb5ad188cb6200cf349068","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-15eb5ad188cb6200cf349068: done: 0.049228 sec","job_status":"done","queue_duration_s":3.571597,"scheduling_latency_s":3.571635,"redis_calls":13,"redis_duration_s":0.014565,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.010761,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049228,"completed_at":"2026-06-17T14:55:05.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002385,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2055vm-test-run-nixbot-gitlab> gitlab # [ 220.981276] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"ab9007daf9d6c4b09c709312","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-ab9007daf9d6c4b09c709312: start","job_status":"start","queue_duration_s":3.629068,"scheduling_latency_s":3.629104}2056vm-test-run-nixbot-gitlab> gitlab # [ 220.982424] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"ab9007daf9d6c4b09c709312","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-ab9007daf9d6c4b09c709312: done: 0.032677 sec","job_status":"done","queue_duration_s":3.629068,"scheduling_latency_s":3.629104,"redis_calls":13,"redis_duration_s":0.01262,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008704,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032677,"completed_at":"2026-06-17T14:55:05.482Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057vm-test-run-nixbot-gitlab> gitlab # [ 221.011120] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3c4aa9534c5e916a287046d5","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3c4aa9534c5e916a287046d5: start","job_status":"start","queue_duration_s":3.66646,"scheduling_latency_s":3.66649}2058vm-test-run-nixbot-gitlab> gitlab # [ 221.032386] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3c4aa9534c5e916a287046d5","created_at":"2026-06-17T14:55:01.738Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-1a733005d84345f9","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"9e105bf4e6a740049c0f11d5a9e9827b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.821Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3c4aa9534c5e916a287046d5: done: 0.034423 sec","job_status":"done","queue_duration_s":3.66646,"scheduling_latency_s":3.66649,"redis_calls":13,"redis_duration_s":0.010963,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034423,"completed_at":"2026-06-17T14:55:05.521Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001706,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2059vm-test-run-nixbot-gitlab> gitlab # [ 221.127110] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.526Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"9dae2ec723529ef540808e7e","created_at":"2026-06-17T14:55:01.858Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db32b105c14f516146ff5450b45c3031","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.989Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-9dae2ec723529ef540808e7e: start","job_status":"start","queue_duration_s":3.536734,"scheduling_latency_s":3.536763}2060vm-test-run-nixbot-gitlab> gitlab # [ 221.128521] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"9dae2ec723529ef540808e7e","created_at":"2026-06-17T14:55:01.858Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db32b105c14f516146ff5450b45c3031","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:01.989Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-9dae2ec723529ef540808e7e: done: 0.055049 sec","job_status":"done","queue_duration_s":3.536734,"scheduling_latency_s":3.536763,"redis_calls":10,"redis_duration_s":0.00881,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00141,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001914,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001483988,"duration_s":0.055049,"completed_at":"2026-06-17T14:55:05.581Z","load_balancing_strategy":"primary","db_duration_s":0.020518,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2061vm-test-run-nixbot-gitlab> gitlab # [ 221.182591] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.586Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0f719e04ae4fde808b7a35ed","created_at":"2026-06-17T14:55:02.026Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43859e0c6395e045404fffb46c2dfb36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.048Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0f719e04ae4fde808b7a35ed: start","job_status":"start","queue_duration_s":3.53798,"scheduling_latency_s":3.538003}2062vm-test-run-nixbot-gitlab> gitlab # [ 221.271210] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.605Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2063vm-test-run-nixbot-gitlab> gitlab # [ 221.272492] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0f719e04ae4fde808b7a35ed","created_at":"2026-06-17T14:55:02.026Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43859e0c6395e045404fffb46c2dfb36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.048Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0f719e04ae4fde808b7a35ed: done: 0.029139 sec","job_status":"done","queue_duration_s":3.53798,"scheduling_latency_s":3.538003,"redis_calls":9,"redis_duration_s":0.008287,"redis_read_bytes":7,"redis_write_bytes":1166,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002204,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":395,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004609,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001474,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029139,"completed_at":"2026-06-17T14:55:05.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001657,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2064vm-test-run-nixbot-gitlab> gitlab # [ 221.276269] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.619Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1567e83e9cc10ea2fcc87ce5","created_at":"2026-06-17T14:55:02.078Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aef8242d7a33b4fb761a6bff5a103b29","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.086Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1567e83e9cc10ea2fcc87ce5: start","job_status":"start","queue_duration_s":3.532839,"scheduling_latency_s":3.532869}2065vm-test-run-nixbot-gitlab> gitlab # [ 221.326281] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.889Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1567e83e9cc10ea2fcc87ce5","created_at":"2026-06-17T14:55:02.078Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aef8242d7a33b4fb761a6bff5a103b29","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.086Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1567e83e9cc10ea2fcc87ce5: done: 0.269437 sec","job_status":"done","queue_duration_s":3.532839,"scheduling_latency_s":3.532869,"redis_calls":9,"redis_duration_s":0.112232,"redis_read_bytes":7,"redis_write_bytes":1237,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":411,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.100157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.010027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.129,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.129,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.269437,"completed_at":"2026-06-17T14:55:05.889Z","load_balancing_strategy":"primary","db_duration_s":0.129281,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2066vm-test-run-nixbot-gitlab> gitlab # [ 221.415671] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.893Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"93dab491adca790eb82df933","created_at":"2026-06-17T14:55:02.213Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50b7cbfe9a33bb7ccc04a7f6ae90f2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-17T14:55:02.228Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-93dab491adca790eb82df933: start","job_status":"start","queue_duration_s":3.664171,"scheduling_latency_s":3.664211}2067vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses2068vm-test-run-nixbot-gitlab> gitlab # [ 221.416984] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.933Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"93dab491adca790eb82df933","created_at":"2026-06-17T14:55:02.213Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50b7cbfe9a33bb7ccc04a7f6ae90f2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-17T14:55:02.228Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-93dab491adca790eb82df933: done: 0.040828 sec","job_status":"done","queue_duration_s":3.664171,"scheduling_latency_s":3.664211,"redis_calls":11,"redis_duration_s":0.011769,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005009,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":658,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00464,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00212,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040828,"completed_at":"2026-06-17T14:55:05.933Z","load_balancing_strategy":"primary","db_duration_s":0.003483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2069vm-test-run-nixbot-gitlab> gitlab # [ 221.452253] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.937Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"f02c610aed84fef86ec7f9c9","created_at":"2026-06-17T14:55:02.257Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d304c2d7a1631d88ed3bf19b654ec0ab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.271Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-f02c610aed84fef86ec7f9c9: start","job_status":"start","queue_duration_s":3.666535,"scheduling_latency_s":3.666561}2070vm-test-run-nixbot-gitlab> gitlab # [ 221.476102] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:05.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"f02c610aed84fef86ec7f9c9","created_at":"2026-06-17T14:55:02.257Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d304c2d7a1631d88ed3bf19b654ec0ab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.271Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-f02c610aed84fef86ec7f9c9: done: 0.060532 sec","job_status":"done","queue_duration_s":3.666535,"scheduling_latency_s":3.666561,"redis_calls":11,"redis_duration_s":0.011670999999999999,"redis_read_bytes":9,"redis_write_bytes":1572,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002389,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":403,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001461918,"exclusive_lock_hold_duration_s":0.017476802,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.060532,"completed_at":"2026-06-17T14:55:05.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019812,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071vm-test-run-nixbot-gitlab> gitlab # [ 221.560230] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"dfb5a2394c6090d68a772282","created_at":"2026-06-17T14:55:02.370Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5e136adde62e0d742a16e7c9f570011","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.382Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-dfb5a2394c6090d68a772282: start","job_status":"start","queue_duration_s":3.619037,"scheduling_latency_s":3.619064}2072vm-test-run-nixbot-gitlab> gitlab # [ 221.569215] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"dfb5a2394c6090d68a772282","created_at":"2026-06-17T14:55:02.370Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5e136adde62e0d742a16e7c9f570011","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.382Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-dfb5a2394c6090d68a772282: done: 0.033728 sec","job_status":"done","queue_duration_s":3.619037,"scheduling_latency_s":3.619064,"redis_calls":9,"redis_duration_s":0.012371,"redis_read_bytes":7,"redis_write_bytes":1246,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003859,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":415,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002079,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033728,"completed_at":"2026-06-17T14:55:06.034Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2073vm-test-run-nixbot-gitlab> gitlab # [ 221.704868] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.038Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"61bd3343ae38acf5695e19c0","created_at":"2026-06-17T14:55:02.401Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"538036608f0147513066bd022e5b4725","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.410Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-61bd3343ae38acf5695e19c0: start","job_status":"start","queue_duration_s":3.627886,"scheduling_latency_s":3.627918}2074vm-test-run-nixbot-gitlab> gitlab # [ 221.706890] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.098Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"61bd3343ae38acf5695e19c0","created_at":"2026-06-17T14:55:02.401Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"538036608f0147513066bd022e5b4725","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.410Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-61bd3343ae38acf5695e19c0: done: 0.059861 sec","job_status":"done","queue_duration_s":3.627886,"scheduling_latency_s":3.627918,"redis_calls":11,"redis_duration_s":0.012789,"redis_read_bytes":9,"redis_write_bytes":1563,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002908,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":401,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005378,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004503,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001632051,"exclusive_lock_hold_duration_s":0.02513392,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.059861,"completed_at":"2026-06-17T14:55:06.098Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2075vm-test-run-nixbot-gitlab> gitlab # [ 221.731175] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0c92c2bf62282d1f22bdb2e6","created_at":"2026-06-17T14:55:02.442Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76127207d2f1e3ef031c52909ee3f715","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.458Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0c92c2bf62282d1f22bdb2e6: start","job_status":"start","queue_duration_s":3.64363,"scheduling_latency_s":3.643653}2076vm-test-run-nixbot-gitlab> gitlab # [ 221.754979] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0c92c2bf62282d1f22bdb2e6","created_at":"2026-06-17T14:55:02.442Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76127207d2f1e3ef031c52909ee3f715","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.458Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0c92c2bf62282d1f22bdb2e6: done: 0.118235 sec","job_status":"done","queue_duration_s":3.64363,"scheduling_latency_s":3.643653,"redis_calls":12,"redis_duration_s":0.011651,"redis_read_bytes":10,"redis_write_bytes":1630,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002403,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":391,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00675,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002498,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.084,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.084,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.083546093,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.118235,"completed_at":"2026-06-17T14:55:06.219Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084047,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2077vm-test-run-nixbot-gitlab> gitlab # [ 221.850650] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.224Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"d015dd2f40e9c1e746026dd7","created_at":"2026-06-17T14:55:02.508Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37b5fa65d1bddbe94afbb071c923a1c7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.796Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-d015dd2f40e9c1e746026dd7: start","job_status":"start","queue_duration_s":3.427665,"scheduling_latency_s":3.427696}2078vm-test-run-nixbot-gitlab> gitlab # [ 221.852058] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"d015dd2f40e9c1e746026dd7","created_at":"2026-06-17T14:55:02.508Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37b5fa65d1bddbe94afbb071c923a1c7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.796Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-d015dd2f40e9c1e746026dd7: done: 0.03307 sec","job_status":"done","queue_duration_s":3.427665,"scheduling_latency_s":3.427696,"redis_calls":11,"redis_duration_s":0.008506,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001385,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001084,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001698,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000944254,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.03307,"completed_at":"2026-06-17T14:55:06.257Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001601,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2079vm-test-run-nixbot-gitlab> gitlab # [ 221.918164] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"136bd1e3ae7eacc590058a43","created_at":"2026-06-17T14:55:02.815Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ce53782cf14bd1834fb8addefd5aed7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.824Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-136bd1e3ae7eacc590058a43: start","job_status":"start","queue_duration_s":3.437538,"scheduling_latency_s":3.437561}2080vm-test-run-nixbot-gitlab> gitlab # [ 221.995381] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"136bd1e3ae7eacc590058a43","created_at":"2026-06-17T14:55:02.815Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ce53782cf14bd1834fb8addefd5aed7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.824Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-136bd1e3ae7eacc590058a43: done: 0.042312 sec","job_status":"done","queue_duration_s":3.437538,"scheduling_latency_s":3.437561,"redis_calls":11,"redis_duration_s":0.011342,"redis_read_bytes":9,"redis_write_bytes":1453,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002506,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":381,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004059,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002196927,"exclusive_lock_hold_duration_s":0.004944763,"duration_s":0.042312,"completed_at":"2026-06-17T14:55:06.303Z","load_balancing_strategy":"primary","db_duration_s":0.003577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081vm-test-run-nixbot-gitlab> gitlab # [ 222.005184] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.308Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"838860bcad57267add5ef2fd","created_at":"2026-06-17T14:55:02.856Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d230084cea961200cc72e09895589511","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.865Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-838860bcad57267add5ef2fd: start","job_status":"start","queue_duration_s":3.442428,"scheduling_latency_s":3.442453}2082vm-test-run-nixbot-gitlab> gitlab # [ 222.028175] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.359Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"838860bcad57267add5ef2fd","created_at":"2026-06-17T14:55:02.856Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d230084cea961200cc72e09895589511","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.865Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-838860bcad57267add5ef2fd: done: 0.051102 sec","job_status":"done","queue_duration_s":3.442428,"scheduling_latency_s":3.442453,"redis_calls":9,"redis_duration_s":0.008779,"redis_read_bytes":7,"redis_write_bytes":1080,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001974,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":369,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001852,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051102,"completed_at":"2026-06-17T14:55:06.359Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2083vm-test-run-nixbot-gitlab> gitlab # [ 222.140365] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.363Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e3bb0f6572f3d295c83e5403","created_at":"2026-06-17T14:55:02.893Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"311e5ac7fa0995f1562e4fb8c46988e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.904Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e3bb0f6572f3d295c83e5403: start","job_status":"start","queue_duration_s":3.458931,"scheduling_latency_s":3.458961}2084vm-test-run-nixbot-gitlab> gitlab # [ 222.141895] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.393Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e3bb0f6572f3d295c83e5403","created_at":"2026-06-17T14:55:02.893Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"311e5ac7fa0995f1562e4fb8c46988e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.904Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e3bb0f6572f3d295c83e5403: done: 0.030841 sec","job_status":"done","queue_duration_s":3.458931,"scheduling_latency_s":3.458961,"redis_calls":9,"redis_duration_s":0.008476,"redis_read_bytes":7,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002291,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":397,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004036,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002149,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030841,"completed_at":"2026-06-17T14:55:06.393Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085vm-test-run-nixbot-gitlab> gitlab # [ 222.185828] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.398Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"293a1f4627d9728f7427f1a1","created_at":"2026-06-17T14:55:02.937Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45abc3ba84417d700d03ceb2f6127461","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.949Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-293a1f4627d9728f7427f1a1: start","job_status":"start","queue_duration_s":3.448532,"scheduling_latency_s":3.448564}2086vm-test-run-nixbot-gitlab> gitlab # [ 222.209166] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.429Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2087vm-test-run-nixbot-gitlab> gitlab # [ 222.215426] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"293a1f4627d9728f7427f1a1","created_at":"2026-06-17T14:55:02.937Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45abc3ba84417d700d03ceb2f6127461","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.949Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-293a1f4627d9728f7427f1a1: done: 0.048606 sec","job_status":"done","queue_duration_s":3.448532,"scheduling_latency_s":3.448564,"redis_calls":9,"redis_duration_s":0.012307,"redis_read_bytes":7,"redis_write_bytes":1232,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001611,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":411,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009235,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001461,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048606,"completed_at":"2026-06-17T14:55:06.446Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001944,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2088vm-test-run-nixbot-gitlab> gitlab # [ 222.286207] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.455Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5d0eb7c78dd5d6449dbec4c7","created_at":"2026-06-17T14:55:02.977Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34228dd63d92d1edf7f59e9042e173f9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.989Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5d0eb7c78dd5d6449dbec4c7: start","job_status":"start","queue_duration_s":3.466578,"scheduling_latency_s":3.46661}2089vm-test-run-nixbot-gitlab> gitlab # [ 222.305088] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5d0eb7c78dd5d6449dbec4c7","created_at":"2026-06-17T14:55:02.977Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34228dd63d92d1edf7f59e9042e173f9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:02.989Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5d0eb7c78dd5d6449dbec4c7: done: 0.035415 sec","job_status":"done","queue_duration_s":3.466578,"scheduling_latency_s":3.46661,"redis_calls":9,"redis_duration_s":0.010342,"redis_read_bytes":7,"redis_write_bytes":1038,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002689,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005569,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002084,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035415,"completed_at":"2026-06-17T14:55:06.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003022,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090vm-test-run-nixbot-gitlab> gitlab # [ 222.413042] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.495Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"081a771b3fe25c9166a19546","created_at":"2026-06-17T14:55:03.122Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6137d986c2ad831a7187b60ff800cb32","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.136Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-081a771b3fe25c9166a19546: start","job_status":"start","queue_duration_s":3.359186,"scheduling_latency_s":3.359229}2091vm-test-run-nixbot-gitlab> gitlab # [ 222.414607] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.910Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"081a771b3fe25c9166a19546","created_at":"2026-06-17T14:55:03.122Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f7e899560af44a5c","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6137d986c2ad831a7187b60ff800cb32","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.136Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-081a771b3fe25c9166a19546: done: 0.415384 sec","job_status":"done","queue_duration_s":3.359186,"scheduling_latency_s":3.359229,"redis_calls":573,"redis_duration_s":0.142293,"redis_read_bytes":571,"redis_write_bytes":49844,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00156,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.13769,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.309459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.415384,"completed_at":"2026-06-17T14:55:06.910Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001407,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2092vm-test-run-nixbot-gitlab> gitlab # [ 222.462440] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2e6929dd0cdb26b2fb77ecd","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2e6929dd0cdb26b2fb77ecd: start","job_status":"start","queue_duration_s":3.66621,"scheduling_latency_s":3.666241}2093vm-test-run-nixbot-gitlab> gitlab # [ 222.482776] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2e6929dd0cdb26b2fb77ecd","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2e6929dd0cdb26b2fb77ecd: done: 0.026647 sec","job_status":"done","queue_duration_s":3.66621,"scheduling_latency_s":3.666241,"redis_calls":23,"redis_duration_s":0.005143,"redis_read_bytes":20,"redis_write_bytes":2604,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002271,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1447,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026647,"completed_at":"2026-06-17T14:55:06.938Z","load_balancing_strategy":"primary","db_duration_s":0.002061,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2094vm-test-run-nixbot-gitlab> gitlab # [ 222.578351] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae37ddc868b6ca1120fbcac1","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae37ddc868b6ca1120fbcac1: start","job_status":"start","queue_duration_s":3.694122,"scheduling_latency_s":3.694144}2095vm-test-run-nixbot-gitlab> gitlab # [ 222.579917] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae37ddc868b6ca1120fbcac1","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae37ddc868b6ca1120fbcac1: done: 0.015038 sec","job_status":"done","queue_duration_s":3.694122,"scheduling_latency_s":3.694144,"redis_calls":13,"redis_duration_s":0.0031929999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015038,"completed_at":"2026-06-17T14:55:06.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2096vm-test-run-nixbot-gitlab> gitlab # [ 222.630693] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb6bb2ccc446062010636f6c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb6bb2ccc446062010636f6c: start","job_status":"start","queue_duration_s":3.710429,"scheduling_latency_s":3.710453}2097vm-test-run-nixbot-gitlab> gitlab # [ 222.650174] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb6bb2ccc446062010636f6c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb6bb2ccc446062010636f6c: done: 0.014605 sec","job_status":"done","queue_duration_s":3.710429,"scheduling_latency_s":3.710453,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014605,"completed_at":"2026-06-17T14:55:06.970Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2098vm-test-run-nixbot-gitlab> gitlab # [ 222.723275] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31586df154ad2698231610d5","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31586df154ad2698231610d5: start","job_status":"start","queue_duration_s":3.726278,"scheduling_latency_s":3.726298}2099vm-test-run-nixbot-gitlab> gitlab # [ 222.731315] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31586df154ad2698231610d5","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31586df154ad2698231610d5: done: 0.014573 sec","job_status":"done","queue_duration_s":3.726278,"scheduling_latency_s":3.726298,"redis_calls":13,"redis_duration_s":0.0029230000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014573,"completed_at":"2026-06-17T14:55:06.986Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100vm-test-run-nixbot-gitlab> gitlab # [ 222.795282] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:06.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ef546b856d40060a9dcf08c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ef546b856d40060a9dcf08c: start","job_status":"start","queue_duration_s":3.742158,"scheduling_latency_s":3.742179}2101vm-test-run-nixbot-gitlab> gitlab # [ 222.867807] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ef546b856d40060a9dcf08c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ef546b856d40060a9dcf08c: done: 0.014817 sec","job_status":"done","queue_duration_s":3.742158,"scheduling_latency_s":3.742179,"redis_calls":13,"redis_duration_s":0.003086,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014817,"completed_at":"2026-06-17T14:55:07.002Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102vm-test-run-nixbot-gitlab> gitlab # [ 222.876228] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56d6e9feaf7c529d23422f4c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56d6e9feaf7c529d23422f4c: start","job_status":"start","queue_duration_s":3.75823,"scheduling_latency_s":3.75825}2103vm-test-run-nixbot-gitlab> gitlab # [ 222.896054] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56d6e9feaf7c529d23422f4c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56d6e9feaf7c529d23422f4c: done: 0.017157 sec","job_status":"done","queue_duration_s":3.75823,"scheduling_latency_s":3.75825,"redis_calls":13,"redis_duration_s":0.003192,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002261,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017157,"completed_at":"2026-06-17T14:55:07.021Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2104vm-test-run-nixbot-gitlab> gitlab # [ 223.012444] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee6b9271170d848ce99d00a6","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee6b9271170d848ce99d00a6: start","job_status":"start","queue_duration_s":3.776932,"scheduling_latency_s":3.776961}2105vm-test-run-nixbot-gitlab> gitlab # [ 223.013732] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee6b9271170d848ce99d00a6","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee6b9271170d848ce99d00a6: done: 0.015369 sec","job_status":"done","queue_duration_s":3.776932,"scheduling_latency_s":3.776961,"redis_calls":13,"redis_duration_s":0.002767,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015369,"completed_at":"2026-06-17T14:55:07.038Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106vm-test-run-nixbot-gitlab> gitlab # [ 223.040829] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cc5a0ceedb18ad9f05d2b66","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cc5a0ceedb18ad9f05d2b66: start","job_status":"start","queue_duration_s":3.793529,"scheduling_latency_s":3.793552}2107vm-test-run-nixbot-gitlab> gitlab # [ 223.060536] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cc5a0ceedb18ad9f05d2b66","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cc5a0ceedb18ad9f05d2b66: done: 0.015688 sec","job_status":"done","queue_duration_s":3.793529,"scheduling_latency_s":3.793552,"redis_calls":13,"redis_duration_s":0.0029709999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001039,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015688,"completed_at":"2026-06-17T14:55:07.055Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2108vm-test-run-nixbot-gitlab> gitlab # [ 223.158319] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28faf33bc0a9d24b1e8335eb","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28faf33bc0a9d24b1e8335eb: start","job_status":"start","queue_duration_s":3.810881,"scheduling_latency_s":3.810909}2109vm-test-run-nixbot-gitlab> gitlab # [ 223.159384] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28faf33bc0a9d24b1e8335eb","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28faf33bc0a9d24b1e8335eb: done: 0.015672 sec","job_status":"done","queue_duration_s":3.810881,"scheduling_latency_s":3.810909,"redis_calls":13,"redis_duration_s":0.0029779999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015672,"completed_at":"2026-06-17T14:55:07.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2110vm-test-run-nixbot-gitlab> gitlab # [ 223.206043] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c02f175957293b1772096d9","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c02f175957293b1772096d9: start","job_status":"start","queue_duration_s":3.827863,"scheduling_latency_s":3.827884}2111vm-test-run-nixbot-gitlab> gitlab # [ 223.225964] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c02f175957293b1772096d9","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c02f175957293b1772096d9: done: 0.015231 sec","job_status":"done","queue_duration_s":3.827863,"scheduling_latency_s":3.827884,"redis_calls":13,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015231,"completed_at":"2026-06-17T14:55:07.088Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2112vm-test-run-nixbot-gitlab> gitlab # [ 223.303195] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67d485b49d29dc47febe8e83","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67d485b49d29dc47febe8e83: start","job_status":"start","queue_duration_s":3.844335,"scheduling_latency_s":3.844355}2113vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 1.95 seconds)2114vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': '2026-06-17T14:55:04.038Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-17T14:55:04.169Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2115vm-test-run-nixbot-gitlab> gitlab # [ 223.307056] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67d485b49d29dc47febe8e83","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67d485b49d29dc47febe8e83: done: 0.015057 sec","job_status":"done","queue_duration_s":3.844335,"scheduling_latency_s":3.844355,"redis_calls":13,"redis_duration_s":0.0030770000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015057,"completed_at":"2026-06-17T14:55:07.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2116vm-test-run-nixbot-gitlab> gitlab # [ 223.371142] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed37813673145aad1bda6357","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed37813673145aad1bda6357: start","job_status":"start","queue_duration_s":3.860679,"scheduling_latency_s":3.860701}2117vm-test-run-nixbot-gitlab> gitlab # [ 223.447270] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed37813673145aad1bda6357","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed37813673145aad1bda6357: done: 0.014767 sec","job_status":"done","queue_duration_s":3.860679,"scheduling_latency_s":3.860701,"redis_calls":13,"redis_duration_s":0.002861,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014767,"completed_at":"2026-06-17T14:55:07.121Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118vm-test-run-nixbot-gitlab> gitlab # [ 223.450959] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f0d0f854f3f19899b7ed7f5","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f0d0f854f3f19899b7ed7f5: start","job_status":"start","queue_duration_s":3.876743,"scheduling_latency_s":3.876767}2119vm-test-run-nixbot-gitlab> gitlab # [ 223.471005] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f0d0f854f3f19899b7ed7f5","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f0d0f854f3f19899b7ed7f5: done: 0.014677 sec","job_status":"done","queue_duration_s":3.876743,"scheduling_latency_s":3.876767,"redis_calls":13,"redis_duration_s":0.0028439999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014677,"completed_at":"2026-06-17T14:55:07.137Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2120vm-test-run-nixbot-gitlab> gitlab # [ 223.591904] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa21dd4ef0ee370a8d3c4661","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa21dd4ef0ee370a8d3c4661: start","job_status":"start","queue_duration_s":3.892716,"scheduling_latency_s":3.892738}2121vm-test-run-nixbot-gitlab> gitlab # [ 223.593815] gitaly[1386]: time="2026-06-17T14:55:07.893Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB1887W5CZ758A9CBF71QN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-17T14:55:37.691" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:07.892" grpc.time_ms=0.835 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2122vm-test-run-nixbot-gitlab> gitlab # [ 223.595271] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 18942123vm-test-run-nixbot-gitlab> gitlab # [ 223.596125] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa21dd4ef0ee370a8d3c4661","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa21dd4ef0ee370a8d3c4661: done: 0.014804 sec","job_status":"done","queue_duration_s":3.892716,"scheduling_latency_s":3.892738,"redis_calls":13,"redis_duration_s":0.0029289999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014804,"completed_at":"2026-06-17T14:55:07.153Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124vm-test-run-nixbot-gitlab> gitlab # [ 223.640204] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92d8288e0ecc2bd8b85166d4","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92d8288e0ecc2bd8b85166d4: start","job_status":"start","queue_duration_s":3.908812,"scheduling_latency_s":3.908834}2125vm-test-run-nixbot-gitlab> gitlab # [ 223.660177] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92d8288e0ecc2bd8b85166d4","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92d8288e0ecc2bd8b85166d4: done: 0.014682 sec","job_status":"done","queue_duration_s":3.908812,"scheduling_latency_s":3.908834,"redis_calls":13,"redis_duration_s":0.002863,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014682,"completed_at":"2026-06-17T14:55:07.169Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2126vm-test-run-nixbot-gitlab> gitlab # [ 223.736096] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bb75184f9a1378b6c4eafd6","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bb75184f9a1378b6c4eafd6: start","job_status":"start","queue_duration_s":3.924827,"scheduling_latency_s":3.924848}2127vm-test-run-nixbot-gitlab> gitlab # [ 223.741168] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bb75184f9a1378b6c4eafd6","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bb75184f9a1378b6c4eafd6: done: 0.066318 sec","job_status":"done","queue_duration_s":3.924827,"scheduling_latency_s":3.924848,"redis_calls":13,"redis_duration_s":0.003044,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.066318,"completed_at":"2026-06-17T14:55:07.236Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2128vm-test-run-nixbot-gitlab> gitlab # [ 223.804152] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97a45749d67b588131044cd1","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97a45749d67b588131044cd1: start","job_status":"start","queue_duration_s":3.993097,"scheduling_latency_s":3.993125}2129vm-test-run-nixbot-gitlab> gitlab # [ 223.880497] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97a45749d67b588131044cd1","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97a45749d67b588131044cd1: done: 0.015541 sec","job_status":"done","queue_duration_s":3.993097,"scheduling_latency_s":3.993125,"redis_calls":13,"redis_duration_s":0.002988,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001018,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015541,"completed_at":"2026-06-17T14:55:07.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2130vm-test-run-nixbot-gitlab> gitlab # [ 223.884760] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8eddd94b02436dbcb120e139","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8eddd94b02436dbcb120e139: start","job_status":"start","queue_duration_s":4.009895,"scheduling_latency_s":4.009915}2131vm-test-run-nixbot-gitlab> gitlab # [ 223.904788] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8eddd94b02436dbcb120e139","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8eddd94b02436dbcb120e139: done: 0.015085 sec","job_status":"done","queue_duration_s":4.009895,"scheduling_latency_s":4.009915,"redis_calls":13,"redis_duration_s":0.002791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015085,"completed_at":"2026-06-17T14:55:07.270Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2132vm-test-run-nixbot-gitlab> gitlab # [ 224.025328] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f7eb92cb87d33654e89578c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f7eb92cb87d33654e89578c: start","job_status":"start","queue_duration_s":4.026777,"scheduling_latency_s":4.026798}2133vm-test-run-nixbot-gitlab> gitlab # [ 224.026849] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f7eb92cb87d33654e89578c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f7eb92cb87d33654e89578c: done: 0.014969 sec","job_status":"done","queue_duration_s":4.026777,"scheduling_latency_s":4.026798,"redis_calls":13,"redis_duration_s":0.002801,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014969,"completed_at":"2026-06-17T14:55:07.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2134vm-test-run-nixbot-gitlab> gitlab # [ 224.049891] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81fbec2ad87912881e43a543","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81fbec2ad87912881e43a543: start","job_status":"start","queue_duration_s":4.043118,"scheduling_latency_s":4.04314}2135vm-test-run-nixbot-gitlab> gitlab # [ 224.069909] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81fbec2ad87912881e43a543","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81fbec2ad87912881e43a543: done: 0.014894 sec","job_status":"done","queue_duration_s":4.043118,"scheduling_latency_s":4.04314,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014894,"completed_at":"2026-06-17T14:55:07.303Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136vm-test-run-nixbot-gitlab> gitlab # [ 224.170179] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a1fc19fd24aa8d01a5b1d80","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a1fc19fd24aa8d01a5b1d80: start","job_status":"start","queue_duration_s":4.059259,"scheduling_latency_s":4.05928}2137vm-test-run-nixbot-gitlab> gitlab # [ 224.171198] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a1fc19fd24aa8d01a5b1d80","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a1fc19fd24aa8d01a5b1d80: done: 0.015477 sec","job_status":"done","queue_duration_s":4.059259,"scheduling_latency_s":4.05928,"redis_calls":13,"redis_duration_s":0.0028710000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015477,"completed_at":"2026-06-17T14:55:07.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2138vm-test-run-nixbot-gitlab> gitlab # [ 224.213858] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2cddfff530854328673989a","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2cddfff530854328673989a: start","job_status":"start","queue_duration_s":4.07599,"scheduling_latency_s":4.076012}2139vm-test-run-nixbot-gitlab> gitlab # [ 224.233443] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2cddfff530854328673989a","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2cddfff530854328673989a: done: 0.014904 sec","job_status":"done","queue_duration_s":4.07599,"scheduling_latency_s":4.076012,"redis_calls":13,"redis_duration_s":0.003223,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002364,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014904,"completed_at":"2026-06-17T14:55:07.336Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140vm-test-run-nixbot-gitlab> gitlab # [ 224.314414] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9271002f6b2e7de07f5f1de1","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9271002f6b2e7de07f5f1de1: start","job_status":"start","queue_duration_s":4.092142,"scheduling_latency_s":4.092163}2141vm-test-run-nixbot-gitlab> gitlab # [ 224.315935] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9271002f6b2e7de07f5f1de1","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9271002f6b2e7de07f5f1de1: done: 0.014969 sec","job_status":"done","queue_duration_s":4.092142,"scheduling_latency_s":4.092163,"redis_calls":13,"redis_duration_s":0.0028580000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014969,"completed_at":"2026-06-17T14:55:07.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2142vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses2143vm-test-run-nixbot-gitlab> gitlab # [ 224.377448] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8585f7697f18900973ba2e30","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8585f7697f18900973ba2e30: start","job_status":"start","queue_duration_s":4.108446,"scheduling_latency_s":4.108468}2144vm-test-run-nixbot-gitlab> gitlab # [ 224.458246] gitaly[1386]: time="2026-06-17T14:55:07.919Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB1887W5CZ758A9CBF71QN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:37.718" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:07.918" grpc.time_ms=0.657 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2145vm-test-run-nixbot-gitlab> gitlab # [ 224.459413] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8585f7697f18900973ba2e30","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8585f7697f18900973ba2e30: done: 0.016562 sec","job_status":"done","queue_duration_s":4.108446,"scheduling_latency_s":4.108468,"redis_calls":13,"redis_duration_s":0.002835,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016562,"completed_at":"2026-06-17T14:55:07.370Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2146vm-test-run-nixbot-gitlab> gitlab # [ 224.477342] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba77095a914e52a837c1043c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba77095a914e52a837c1043c: start","job_status":"start","queue_duration_s":4.126304,"scheduling_latency_s":4.126327}2147vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.12 seconds)2148vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': '2026-06-17T14:55:04.038Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-17T14:55:04.169Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2149vm-test-run-nixbot-gitlab> gitlab # [ 224.497457] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba77095a914e52a837c1043c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba77095a914e52a837c1043c: done: 0.014944 sec","job_status":"done","queue_duration_s":4.126304,"scheduling_latency_s":4.126327,"redis_calls":13,"redis_duration_s":0.0028339999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014944,"completed_at":"2026-06-17T14:55:07.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2150vm-test-run-nixbot-gitlab> gitlab # [ 224.602199] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7259910fdb19a9f3f13716b3","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7259910fdb19a9f3f13716b3: start","job_status":"start","queue_duration_s":4.143072,"scheduling_latency_s":4.143093}2151vm-test-run-nixbot-gitlab> gitlab # [ 224.603430] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7259910fdb19a9f3f13716b3","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7259910fdb19a9f3f13716b3: done: 0.015546 sec","job_status":"done","queue_duration_s":4.143072,"scheduling_latency_s":4.143093,"redis_calls":13,"redis_duration_s":0.003052,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015546,"completed_at":"2026-06-17T14:55:07.404Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2152vm-test-run-nixbot-gitlab> gitlab # [ 224.642070] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9d8af4e4183c0bfa25dddf4","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9d8af4e4183c0bfa25dddf4: start","job_status":"start","queue_duration_s":4.15989,"scheduling_latency_s":4.159911}2153vm-test-run-nixbot-gitlab> gitlab # [ 224.662107] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9d8af4e4183c0bfa25dddf4","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9d8af4e4183c0bfa25dddf4: done: 0.014873 sec","job_status":"done","queue_duration_s":4.15989,"scheduling_latency_s":4.159911,"redis_calls":13,"redis_duration_s":0.002833,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014873,"completed_at":"2026-06-17T14:55:07.420Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154vm-test-run-nixbot-gitlab> gitlab # [ 224.747288] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ba02011c10b18fe414ba934","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ba02011c10b18fe414ba934: start","job_status":"start","queue_duration_s":4.175996,"scheduling_latency_s":4.176017}2155vm-test-run-nixbot-gitlab> gitlab # [ 224.748337] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ba02011c10b18fe414ba934","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ba02011c10b18fe414ba934: done: 0.015705 sec","job_status":"done","queue_duration_s":4.175996,"scheduling_latency_s":4.176017,"redis_calls":13,"redis_duration_s":0.003405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015705,"completed_at":"2026-06-17T14:55:07.437Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156vm-test-run-nixbot-gitlab> gitlab # [ 224.806143] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b8c077f5cdc99a787db0d8a","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b8c077f5cdc99a787db0d8a: start","job_status":"start","queue_duration_s":4.192941,"scheduling_latency_s":4.192962}2157vm-test-run-nixbot-gitlab> gitlab # [ 224.890882] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b8c077f5cdc99a787db0d8a","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b8c077f5cdc99a787db0d8a: done: 0.015169 sec","job_status":"done","queue_duration_s":4.192941,"scheduling_latency_s":4.192962,"redis_calls":13,"redis_duration_s":0.0029379999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015169,"completed_at":"2026-06-17T14:55:07.453Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2158vm-test-run-nixbot-gitlab> gitlab # [ 224.893099] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9564bc26e3a06e29732e6a7f","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9564bc26e3a06e29732e6a7f: start","job_status":"start","queue_duration_s":4.210149,"scheduling_latency_s":4.210192}2159vm-test-run-nixbot-gitlab> gitlab # [ 224.906242] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9564bc26e3a06e29732e6a7f","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9564bc26e3a06e29732e6a7f: done: 0.019689 sec","job_status":"done","queue_duration_s":4.210149,"scheduling_latency_s":4.210192,"redis_calls":13,"redis_duration_s":0.00301,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019689,"completed_at":"2026-06-17T14:55:07.475Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160vm-test-run-nixbot-gitlab> gitlab # [ 225.034191] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b417335a4e794df1271c971a","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b417335a4e794df1271c971a: start","job_status":"start","queue_duration_s":4.231041,"scheduling_latency_s":4.231063}2161vm-test-run-nixbot-gitlab> gitlab # [ 225.035350] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b417335a4e794df1271c971a","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b417335a4e794df1271c971a: done: 0.015038 sec","job_status":"done","queue_duration_s":4.231041,"scheduling_latency_s":4.231063,"redis_calls":13,"redis_duration_s":0.002818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015038,"completed_at":"2026-06-17T14:55:07.491Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2162vm-test-run-nixbot-gitlab> gitlab # [ 225.049878] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c016fab12f5ec29a31323ab1","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c016fab12f5ec29a31323ab1: start","job_status":"start","queue_duration_s":4.247344,"scheduling_latency_s":4.247365}2163vm-test-run-nixbot-gitlab> gitlab # [ 225.069850] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c016fab12f5ec29a31323ab1","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c016fab12f5ec29a31323ab1: done: 0.015894 sec","job_status":"done","queue_duration_s":4.247344,"scheduling_latency_s":4.247365,"redis_calls":13,"redis_duration_s":0.003667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002279,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015894,"completed_at":"2026-06-17T14:55:07.509Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164vm-test-run-nixbot-gitlab> gitlab # [ 225.178278] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"507c59a6e10d618f0d37ed93","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-507c59a6e10d618f0d37ed93: start","job_status":"start","queue_duration_s":4.264478,"scheduling_latency_s":4.264498}2165vm-test-run-nixbot-gitlab> gitlab # [ 225.179530] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"507c59a6e10d618f0d37ed93","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-507c59a6e10d618f0d37ed93: done: 0.015103 sec","job_status":"done","queue_duration_s":4.264478,"scheduling_latency_s":4.264498,"redis_calls":13,"redis_duration_s":0.002818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015103,"completed_at":"2026-06-17T14:55:07.525Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166vm-test-run-nixbot-gitlab> gitlab # [ 225.213189] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62cad770bd2fdab6f5bcbe8c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62cad770bd2fdab6f5bcbe8c: start","job_status":"start","queue_duration_s":4.280848,"scheduling_latency_s":4.280869}2167vm-test-run-nixbot-gitlab> gitlab # [ 225.233720] gitaly[1386]: time="2026-06-17T14:55:07.952Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB1887W5CZ758A9CBF71QN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:37.750" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:07.951" grpc.time_ms=0.818 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2168vm-test-run-nixbot-gitlab> gitlab # [ 225.253556] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 592169vm-test-run-nixbot-gitlab> gitlab # [ 225.322382] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62cad770bd2fdab6f5bcbe8c","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62cad770bd2fdab6f5bcbe8c: done: 0.015536 sec","job_status":"done","queue_duration_s":4.280848,"scheduling_latency_s":4.280869,"redis_calls":13,"redis_duration_s":0.00285,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015536,"completed_at":"2026-06-17T14:55:07.542Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2170vm-test-run-nixbot-gitlab> gitlab # [ 225.324192] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5466d0eb644dbdacf879abc","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5466d0eb644dbdacf879abc: start","job_status":"start","queue_duration_s":4.297687,"scheduling_latency_s":4.29771}2171vm-test-run-nixbot-gitlab> gitlab # [ 225.337894] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5466d0eb644dbdacf879abc","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5466d0eb644dbdacf879abc: done: 0.01512 sec","job_status":"done","queue_duration_s":4.297687,"scheduling_latency_s":4.29771,"redis_calls":13,"redis_duration_s":0.003103,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002175,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01512,"completed_at":"2026-06-17T14:55:07.558Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172vm-test-run-nixbot-gitlab> gitlab # [ 225.465979] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ae544c54293d1d7ba334f92","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ae544c54293d1d7ba334f92: start","job_status":"start","queue_duration_s":4.314095,"scheduling_latency_s":4.314116}2173vm-test-run-nixbot-gitlab> gitlab # [ 225.467265] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ae544c54293d1d7ba334f92","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ae544c54293d1d7ba334f92: done: 0.015708 sec","job_status":"done","queue_duration_s":4.314095,"scheduling_latency_s":4.314116,"redis_calls":13,"redis_duration_s":0.003011,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015708,"completed_at":"2026-06-17T14:55:07.575Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2174vm-test-run-nixbot-gitlab> gitlab # [ 225.480935] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200bf7ebcd9efa72db5d0691","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200bf7ebcd9efa72db5d0691: start","job_status":"start","queue_duration_s":4.331132,"scheduling_latency_s":4.331153}2175vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses2176vm-test-run-nixbot-gitlab> gitlab # [ 225.500849] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200bf7ebcd9efa72db5d0691","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200bf7ebcd9efa72db5d0691: done: 0.015365 sec","job_status":"done","queue_duration_s":4.331132,"scheduling_latency_s":4.331153,"redis_calls":13,"redis_duration_s":0.002806,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015365,"completed_at":"2026-06-17T14:55:07.592Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2177vm-test-run-nixbot-gitlab> gitlab # [ 225.609287] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31e08383bf3f208c6f0d4915","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31e08383bf3f208c6f0d4915: start","job_status":"start","queue_duration_s":4.347796,"scheduling_latency_s":4.347817}2178vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.09 seconds)2179vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': '2026-06-17T14:55:04.038Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-17T14:55:04.169Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2180vm-test-run-nixbot-gitlab> gitlab # [ 225.610817] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31e08383bf3f208c6f0d4915","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31e08383bf3f208c6f0d4915: done: 0.017898 sec","job_status":"done","queue_duration_s":4.347796,"scheduling_latency_s":4.347817,"redis_calls":13,"redis_duration_s":0.003747,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002787,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017898,"completed_at":"2026-06-17T14:55:07.611Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2181vm-test-run-nixbot-gitlab> gitlab # [ 225.643745] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9371de80c17127f06cd1cde8","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9371de80c17127f06cd1cde8: start","job_status":"start","queue_duration_s":4.367857,"scheduling_latency_s":4.367886}2182vm-test-run-nixbot-gitlab> gitlab # [ 225.663533] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9371de80c17127f06cd1cde8","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9371de80c17127f06cd1cde8: done: 0.016078 sec","job_status":"done","queue_duration_s":4.367857,"scheduling_latency_s":4.367886,"redis_calls":13,"redis_duration_s":0.002813,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016078,"completed_at":"2026-06-17T14:55:07.629Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2183vm-test-run-nixbot-gitlab> gitlab # [ 225.754085] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ac6a309cd87f6e11d0be770","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ac6a309cd87f6e11d0be770: start","job_status":"start","queue_duration_s":4.385156,"scheduling_latency_s":4.385179}2184vm-test-run-nixbot-gitlab> gitlab # [ 225.755345] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ac6a309cd87f6e11d0be770","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ac6a309cd87f6e11d0be770: done: 0.015555 sec","job_status":"done","queue_duration_s":4.385156,"scheduling_latency_s":4.385179,"redis_calls":13,"redis_duration_s":0.00277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015555,"completed_at":"2026-06-17T14:55:07.646Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2185vm-test-run-nixbot-gitlab> gitlab # [ 225.807374] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"687a20aab63a02237ca22edf","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-687a20aab63a02237ca22edf: start","job_status":"start","queue_duration_s":4.401962,"scheduling_latency_s":4.401982}2186vm-test-run-nixbot-gitlab> gitlab # [ 225.827243] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"687a20aab63a02237ca22edf","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-687a20aab63a02237ca22edf: done: 0.015001 sec","job_status":"done","queue_duration_s":4.401962,"scheduling_latency_s":4.401982,"redis_calls":13,"redis_duration_s":0.002794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015001,"completed_at":"2026-06-17T14:55:07.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187vm-test-run-nixbot-gitlab> gitlab # [ 225.898491] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a00a14ce3672d4652b59e6b2","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a00a14ce3672d4652b59e6b2: start","job_status":"start","queue_duration_s":4.418211,"scheduling_latency_s":4.418232}2188vm-test-run-nixbot-gitlab> gitlab # [ 225.907242] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a00a14ce3672d4652b59e6b2","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a00a14ce3672d4652b59e6b2: done: 0.014997 sec","job_status":"done","queue_duration_s":4.418211,"scheduling_latency_s":4.418232,"redis_calls":13,"redis_duration_s":0.002841,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014997,"completed_at":"2026-06-17T14:55:07.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2189vm-test-run-nixbot-gitlab> gitlab # [ 225.970555] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea16108d77e683a4ff6b83cd","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea16108d77e683a4ff6b83cd: start","job_status":"start","queue_duration_s":4.434449,"scheduling_latency_s":4.43447}2190vm-test-run-nixbot-gitlab> gitlab # [ 226.041135] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea16108d77e683a4ff6b83cd","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea16108d77e683a4ff6b83cd: done: 0.017348 sec","job_status":"done","queue_duration_s":4.434449,"scheduling_latency_s":4.43447,"redis_calls":13,"redis_duration_s":0.0031729999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017348,"completed_at":"2026-06-17T14:55:07.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2191vm-test-run-nixbot-gitlab> gitlab # [ 226.051240] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6318c41575ec04524ba0210b","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6318c41575ec04524ba0210b: start","job_status":"start","queue_duration_s":4.454265,"scheduling_latency_s":4.454295}2192vm-test-run-nixbot-gitlab> gitlab # [ 226.071352] gitaly[1386]: time="2026-06-17T14:55:07.969Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB18ABE0M1T9GQ3JBFQEB5Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:37.368" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:07.968" grpc.time_ms=0.644 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2193vm-test-run-nixbot-gitlab> gitlab # [ 226.091813] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 522194vm-test-run-nixbot-gitlab> gitlab # [ 226.095790] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6318c41575ec04524ba0210b","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6318c41575ec04524ba0210b: done: 0.016164 sec","job_status":"done","queue_duration_s":4.454265,"scheduling_latency_s":4.454295,"redis_calls":13,"redis_duration_s":0.002904,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016164,"completed_at":"2026-06-17T14:55:07.716Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195vm-test-run-nixbot-gitlab> gitlab # [ 226.185196] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbe2c1d1575cc3ea39f60072","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbe2c1d1575cc3ea39f60072: start","job_status":"start","queue_duration_s":4.471728,"scheduling_latency_s":4.47175}2196vm-test-run-nixbot-gitlab> gitlab # [ 226.186425] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbe2c1d1575cc3ea39f60072","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbe2c1d1575cc3ea39f60072: done: 0.014772 sec","job_status":"done","queue_duration_s":4.471728,"scheduling_latency_s":4.47175,"redis_calls":13,"redis_duration_s":0.0031509999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014772,"completed_at":"2026-06-17T14:55:07.732Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197vm-test-run-nixbot-gitlab> gitlab # [ 226.239241] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab317e0267169d28b558dd36","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab317e0267169d28b558dd36: start","job_status":"start","queue_duration_s":4.488764,"scheduling_latency_s":4.488785}2198vm-test-run-nixbot-gitlab> gitlab # [ 226.259105] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab317e0267169d28b558dd36","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab317e0267169d28b558dd36: done: 0.017405 sec","job_status":"done","queue_duration_s":4.488764,"scheduling_latency_s":4.488785,"redis_calls":13,"redis_duration_s":0.0030819999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017405,"completed_at":"2026-06-17T14:55:07.752Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199vm-test-run-nixbot-gitlab> gitlab # [ 226.329156] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23bc3bbaa7144646452616ac","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23bc3bbaa7144646452616ac: start","job_status":"start","queue_duration_s":4.507512,"scheduling_latency_s":4.507533}2200vm-test-run-nixbot-gitlab> gitlab # [ 226.339521] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23bc3bbaa7144646452616ac","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23bc3bbaa7144646452616ac: done: 0.015503 sec","job_status":"done","queue_duration_s":4.507512,"scheduling_latency_s":4.507533,"redis_calls":13,"redis_duration_s":0.002867,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015503,"completed_at":"2026-06-17T14:55:07.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201vm-test-run-nixbot-gitlab> gitlab # [ 226.402939] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8824813407789ff3d9e54746","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8824813407789ff3d9e54746: start","job_status":"start","queue_duration_s":4.524271,"scheduling_latency_s":4.524292}2202vm-test-run-nixbot-gitlab> gitlab # [ 226.472249] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8824813407789ff3d9e54746","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8824813407789ff3d9e54746: done: 0.018374 sec","job_status":"done","queue_duration_s":4.524271,"scheduling_latency_s":4.524292,"redis_calls":13,"redis_duration_s":0.0030759999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018374,"completed_at":"2026-06-17T14:55:07.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203vm-test-run-nixbot-gitlab> gitlab # [ 226.483122] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e9204bd8998ba1628583af2","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e9204bd8998ba1628583af2: start","job_status":"start","queue_duration_s":4.544185,"scheduling_latency_s":4.544216}2204vm-test-run-nixbot-gitlab> gitlab # [ 226.503315] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e9204bd8998ba1628583af2","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e9204bd8998ba1628583af2: done: 0.016496 sec","job_status":"done","queue_duration_s":4.544185,"scheduling_latency_s":4.544216,"redis_calls":13,"redis_duration_s":0.002981,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016496,"completed_at":"2026-06-17T14:55:07.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2205vm-test-run-nixbot-gitlab> gitlab # [ 226.616151] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e012aec24f9489123c0f8383","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e012aec24f9489123c0f8383: start","job_status":"start","queue_duration_s":4.561952,"scheduling_latency_s":4.561973}2206vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses2207vm-test-run-nixbot-gitlab> gitlab # [ 226.617165] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e012aec24f9489123c0f8383","created_at":"2026-06-17T14:55:02.608Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-38112bcbc4d74f3d","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5ed907032e7ba58c2f7b97521581b64e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:03.245Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e012aec24f9489123c0f8383: done: 0.014879 sec","job_status":"done","queue_duration_s":4.561952,"scheduling_latency_s":4.561973,"redis_calls":13,"redis_duration_s":0.003268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002375,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014879,"completed_at":"2026-06-17T14:55:07.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2208vm-test-run-nixbot-gitlab> gitlab # [ 226.646937] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.824Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6ef0c63556aa556913ccf3cf","created_at":"2026-06-17T14:55:04.046Z","trace_propagation_headers":{"sentry-trace":"3d5954bc96944e47b96a820a39ee3c38-7f61a57fb0d34eb3","baggage":"sentry-trace_id=3d5954bc96944e47b96a820a39ee3c38,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB1887W5CZ758A9CBF71QN0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.049Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6ef0c63556aa556913ccf3cf: start","job_status":"start","queue_duration_s":3.774098,"scheduling_latency_s":3.774119}2209vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.10 seconds)2210vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': '2026-06-17T14:55:04.038Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-17T14:55:04.169Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2211vm-test-run-nixbot-gitlab> gitlab # [ 226.671723] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.961Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6ef0c63556aa556913ccf3cf","created_at":"2026-06-17T14:55:04.046Z","trace_propagation_headers":{"sentry-trace":"3d5954bc96944e47b96a820a39ee3c38-7f61a57fb0d34eb3","baggage":"sentry-trace_id=3d5954bc96944e47b96a820a39ee3c38,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB1887W5CZ758A9CBF71QN0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.049Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2150,"job_status":"rescheduled","message":"PipelineProcessWorker JID-6ef0c63556aa556913ccf3cf: rescheduled"}2212vm-test-run-nixbot-gitlab> gitlab # [ 226.759599] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.970Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6ef0c63556aa556913ccf3cf","created_at":"2026-06-17T14:55:04.046Z","trace_propagation_headers":{"sentry-trace":"3d5954bc96944e47b96a820a39ee3c38-7f61a57fb0d34eb3","baggage":"sentry-trace_id=3d5954bc96944e47b96a820a39ee3c38,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB1887W5CZ758A9CBF71QN0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.049Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6ef0c63556aa556913ccf3cf: done: 0.146751 sec","job_status":"done","queue_duration_s":3.774098,"scheduling_latency_s":3.774119,"gitaly_calls":3,"gitaly_duration_s":0.010994,"redis_calls":53,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.014884,"redis_read_bytes":109,"redis_write_bytes":8996,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001041,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.006806,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_calls":6,"redis_queues_duration_s":0.000933,"redis_queues_read_bytes":6,"redis_queues_write_bytes":3276,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00242,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1187,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000455,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000483,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001233,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095256,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.116753107,"duration_s":0.146751,"completed_at":"2026-06-17T14:55:07.970Z","load_balancing_strategy":"primary","db_duration_s":0.03716,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2213vm-test-run-nixbot-gitlab> gitlab # [ 226.781541] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.972Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"272ce3b63b269b477440a63e","created_at":"2026-06-17T14:55:04.338Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f2b9a5091def4b94","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.372Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-272ce3b63b269b477440a63e: start","job_status":"start","queue_duration_s":3.600457,"scheduling_latency_s":3.600488}2214vm-test-run-nixbot-gitlab> gitlab # [ 226.801491] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.990Z","message":"No existing merge request to be cleaned up."}2215vm-test-run-nixbot-gitlab> gitlab # [ 226.803979] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.998Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"272ce3b63b269b477440a63e","created_at":"2026-06-17T14:55:04.338Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f2b9a5091def4b94","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.372Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-272ce3b63b269b477440a63e: done: 0.025639 sec","job_status":"done","queue_duration_s":3.600457,"scheduling_latency_s":3.600488,"redis_calls":14,"redis_duration_s":0.003962,"redis_read_bytes":12,"redis_write_bytes":1678,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000952,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001981,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":5,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025639,"completed_at":"2026-06-17T14:55:07.998Z","load_balancing_strategy":"primary","db_duration_s":0.003692,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216vm-test-run-nixbot-gitlab> gitlab # [ 226.907836] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:07.999Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f3f6c1cf3a7f339341c4bf5d","created_at":"2026-06-17T14:55:04.338Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f2b9a5091def4b94","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.372Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f3f6c1cf3a7f339341c4bf5d: start","job_status":"start","queue_duration_s":3.627435,"scheduling_latency_s":3.627464}2217vm-test-run-nixbot-gitlab> gitlab # [ 226.909185] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.009Z","message":"No existing merge request to be cleaned up."}2218vm-test-run-nixbot-gitlab> gitlab # [ 226.909984] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.015Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f3f6c1cf3a7f339341c4bf5d","created_at":"2026-06-17T14:55:04.338Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f2b9a5091def4b94","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.372Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f3f6c1cf3a7f339341c4bf5d: done: 0.016012 sec","job_status":"done","queue_duration_s":3.627435,"scheduling_latency_s":3.627464,"redis_calls":10,"redis_duration_s":0.002295,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001372,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016012,"completed_at":"2026-06-17T14:55:08.015Z","load_balancing_strategy":"primary","db_duration_s":0.001839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2219vm-test-run-nixbot-gitlab> gitlab # [ 226.951285] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.017Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3dcc6af40a3e3384a49dac53","created_at":"2026-06-17T14:55:04.338Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f2b9a5091def4b94","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.372Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3dcc6af40a3e3384a49dac53: start","job_status":"start","queue_duration_s":3.644683,"scheduling_latency_s":3.644704}2220vm-test-run-nixbot-gitlab> gitlab # [ 226.971177] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.026Z","message":"No existing merge request to be cleaned up."}2221vm-test-run-nixbot-gitlab> gitlab # [ 226.974270] gitaly[1386]: time="2026-06-17T14:55:09.113Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB18D8H7XJBMMAAFTH765PN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:39.111" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:09.112" grpc.time_ms=0.655 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2222vm-test-run-nixbot-gitlab> gitlab # [ 227.051588] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 522223vm-test-run-nixbot-gitlab> gitlab # [ 227.053074] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.032Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3dcc6af40a3e3384a49dac53","created_at":"2026-06-17T14:55:04.338Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f2b9a5091def4b94","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.372Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3dcc6af40a3e3384a49dac53: done: 0.015397 sec","job_status":"done","queue_duration_s":3.644683,"scheduling_latency_s":3.644704,"redis_calls":10,"redis_duration_s":0.002257,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001371,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015397,"completed_at":"2026-06-17T14:55:08.032Z","load_balancing_strategy":"primary","db_duration_s":0.001529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224vm-test-run-nixbot-gitlab> gitlab # [ 227.058196] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.033Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b30b6b2c7443281af85883d2","created_at":"2026-06-17T14:55:04.338Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f2b9a5091def4b94","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.372Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b30b6b2c7443281af85883d2: start","job_status":"start","queue_duration_s":3.661389,"scheduling_latency_s":3.661411}2225vm-test-run-nixbot-gitlab> gitlab # [ 227.077844] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.043Z","message":"No existing merge request to be cleaned up."}2226vm-test-run-nixbot-gitlab> gitlab # [ 227.080088] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.048Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b30b6b2c7443281af85883d2","created_at":"2026-06-17T14:55:04.338Z","trace_propagation_headers":{"sentry-trace":"67110e3ab77f40d4ba2ed60b358d037a-f2b9a5091def4b94","baggage":"sentry-trace_id=67110e3ab77f40d4ba2ed60b358d037a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0a0080c40b1e2508efc063f3b7f3f069","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-17T14:55:04.372Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b30b6b2c7443281af85883d2: done: 0.015098 sec","job_status":"done","queue_duration_s":3.661389,"scheduling_latency_s":3.661411,"redis_calls":10,"redis_duration_s":0.002247,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001352,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015098,"completed_at":"2026-06-17T14:55:08.048Z","load_balancing_strategy":"primary","db_duration_s":0.001575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2227vm-test-run-nixbot-gitlab> gitlab # [ 227.194889] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.050Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"bc803128d94261793a70cf36","created_at":"2026-06-17T14:55:07.878Z","trace_propagation_headers":{"sentry-trace":"3d5954bc96944e47b96a820a39ee3c38-3deb2b752b9e4776","baggage":"sentry-trace_id=3d5954bc96944e47b96a820a39ee3c38,sentry-release=Unknown"},"correlation_id":"01KVB1887W5CZ758A9CBF71QN0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:07.880Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bc803128d94261793a70cf36: start","job_status":"start","queue_duration_s":0.169723,"scheduling_latency_s":0.169747}2228vm-test-run-nixbot-gitlab> gitlab # [ 227.196262] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.075Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"bc803128d94261793a70cf36","created_at":"2026-06-17T14:55:07.878Z","trace_propagation_headers":{"sentry-trace":"3d5954bc96944e47b96a820a39ee3c38-3deb2b752b9e4776","baggage":"sentry-trace_id=3d5954bc96944e47b96a820a39ee3c38,sentry-release=Unknown"},"correlation_id":"01KVB1887W5CZ758A9CBF71QN0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:07.880Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bc803128d94261793a70cf36: done: 0.024866 sec","job_status":"done","queue_duration_s":0.169723,"scheduling_latency_s":0.169747,"redis_calls":7,"redis_duration_s":0.0016229999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001187,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024866,"completed_at":"2026-06-17T14:55:08.075Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004054,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2229vm-test-run-nixbot-gitlab> gitlab # [ 227.234214] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.076Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"fdd3ccc7c59d8b1d716ce46d","created_at":"2026-06-17T14:55:07.921Z","trace_propagation_headers":{"sentry-trace":"3d5954bc96944e47b96a820a39ee3c38-3deb2b752b9e4776","baggage":"sentry-trace_id=3d5954bc96944e47b96a820a39ee3c38,sentry-release=Unknown"},"correlation_id":"01KVB1887W5CZ758A9CBF71QN0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:07.925Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-fdd3ccc7c59d8b1d716ce46d: start","job_status":"start","queue_duration_s":0.15164,"scheduling_latency_s":0.151662}2230vm-test-run-nixbot-gitlab> gitlab # [ 227.259055] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.101Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"fdd3ccc7c59d8b1d716ce46d","created_at":"2026-06-17T14:55:07.921Z","trace_propagation_headers":{"sentry-trace":"3d5954bc96944e47b96a820a39ee3c38-3deb2b752b9e4776","baggage":"sentry-trace_id=3d5954bc96944e47b96a820a39ee3c38,sentry-release=Unknown"},"correlation_id":"01KVB1887W5CZ758A9CBF71QN0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:07.925Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-fdd3ccc7c59d8b1d716ce46d: done: 0.024427 sec","job_status":"done","queue_duration_s":0.15164,"scheduling_latency_s":0.151662,"redis_calls":10,"redis_duration_s":0.002313,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000447,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346133,"duration_s":0.024427,"completed_at":"2026-06-17T14:55:08.101Z","load_balancing_strategy":"primary","db_duration_s":0.003769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231vm-test-run-nixbot-gitlab> gitlab # [ 227.339278] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.102Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"db5a717996ed27e9997d2e05","created_at":"2026-06-17T14:55:07.961Z","trace_propagation_headers":{"sentry-trace":"3d5954bc96944e47b96a820a39ee3c38-3deb2b752b9e4776","baggage":"sentry-trace_id=3d5954bc96944e47b96a820a39ee3c38,sentry-release=Unknown"},"correlation_id":"01KVB1887W5CZ758A9CBF71QN0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:07.963Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-db5a717996ed27e9997d2e05: start","job_status":"start","queue_duration_s":0.139057,"scheduling_latency_s":0.139077}2232vm-test-run-nixbot-gitlab> gitlab # [ 227.351553] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:08.118Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"db5a717996ed27e9997d2e05","created_at":"2026-06-17T14:55:07.961Z","trace_propagation_headers":{"sentry-trace":"3d5954bc96944e47b96a820a39ee3c38-3deb2b752b9e4776","baggage":"sentry-trace_id=3d5954bc96944e47b96a820a39ee3c38,sentry-release=Unknown"},"correlation_id":"01KVB1887W5CZ758A9CBF71QN0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:07.963Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-db5a717996ed27e9997d2e05: done: 0.016085 sec","job_status":"done","queue_duration_s":0.139057,"scheduling_latency_s":0.139077,"redis_calls":8,"redis_duration_s":0.001922,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001471,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016085,"completed_at":"2026-06-17T14:55:08.118Z","load_balancing_strategy":"primary","db_duration_s":0.001278,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2233vm-test-run-nixbot-gitlab> gitlab # [ 227.483612] gitaly[1386]: time="2026-06-17T14:55:10.203Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB18EAQVE3JG5ZYSX2S24JS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:40.102" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:10.202" grpc.time_ms=0.651 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2234vm-test-run-nixbot-gitlab> gitlab # [ 227.484759] gitaly[1386]: time="2026-06-17T14:55:11.306Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB18FD660AKBQ3RAF05C4JM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:41.204" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:11.305" grpc.time_ms=0.964 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2235vm-test-run-nixbot-gitlab> gitlab # [ 227.791382] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 5285749874 wd_nsec: 52857469042236vm-test-run-nixbot-gitlab> gitlab # [ 227.598190] nix-daemon[3447]: accepted connection from pid 3681, user nixbot2237vm-test-run-nixbot-gitlab> gitlab # [ 227.670942] gitaly[1386]: time="2026-06-17T14:55:12.233Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB18G9RTZNX6402YVXXDXFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:42.131" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:12.232" grpc.time_ms=1.358 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2238vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses2239vm-test-run-nixbot-gitlab> gitlab # [ 227.827254] gitaly[1386]: time="2026-06-17T14:55:12.389Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KVB18G9RTZNX6402YVXXDXFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-17T14:55:22.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-17T14:55:12.377" grpc.time_ms=12.254 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2240vm-test-run-nixbot-gitlab> gitlab # [ 227.873757] gitaly[1386]: time="2026-06-17T14:55:12.436Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KVB18GFJ0VCJP0FZPDWYC3BT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-17T14:55:42.435" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-17T14:55:12.436" grpc.time_ms=0.747 pid=1386 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2241vm-test-run-nixbot-gitlab> gitlab # [ 227.880681] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-06-17T14:55:12.443Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"12ab62a090f3ab14be2e76f0","created_at":"2026-06-17T14:55:12.434Z","trace_propagation_headers":{"sentry-trace":"8c42fb1bcc874a77af966703a9c5fc56-3c9f9b2173334ea8","baggage":"sentry-trace_id=8c42fb1bcc874a77af966703a9c5fc56,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KVB18G9RTZNX6402YVXXDXFJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-17T14:55:12.440Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m5zi","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-12ab62a090f3ab14be2e76f0: start","job_status":"start","queue_duration_s":0.002598,"scheduling_latency_s":0.002638}2242vm-test-run-nixbot-gitlab> gitlab # [ 227.894570] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/14b3955733a0a15351b056a59a904cda74dfdf9d HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2722243vm-test-run-nixbot-gitlab> gitlab # [ 227.898229] nixbot[3173]: {"ts": "2026-06-17T14:55:12.458Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/14b3955733a0a15351b056a59a904cda74dfdf9d \"HTTP/1.1 201 Created\""}2244vm-test-run-nixbot-gitlab> gitlab # [ 227.948599] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/06/17:14:55:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 1402245vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/14b3955733a0a15351b056a59a904cda74dfdf9d/statuses, in 0.19 seconds)2246vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-17T14:54:56.822Z', 'started_at': None, 'finished_at': '2026-06-17T14:55:04.038Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '14b3955733a0a15351b056a59a904cda74dfdf9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-06-17T14:55:04.169Z', 'started_at': None, 'finished_at': '2026-06-17T14:55:12.424Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2247vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 16.32 seconds)2248vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 263.30 seconds)2249vm-test-run-nixbot-gitlab> test script finished in 263.33s2250vm-test-run-nixbot-gitlab> cleanup2251vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2252vm-test-run-nixbot-gitlab> gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2253vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.54 seconds)22542255post-build step Upload coverage to codecov: ok2256Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab