1these 13 derivations will be built:2 /nix/store/2vzniwmy0apswgmrcx4ajm0n8zh0hpjf-python3.13-nixbot.drv3 /nix/store/dbhidf525lg7pm7g8iknm0xfdbl3jfwz-unit-nixbot.service.drv4 /nix/store/dw022wlrcp6hl42yxc794yhb7p1kqj52-system-units.drv5 /nix/store/81jyf6f1dpbrvx840wzx9gxig2b95zxw-etc.drv6 /nix/store/vfclw52j9lh2h4g59zfk8jlssmx5wv55-activate.drv7 /nix/store/q1iysgrk1dbzjq8hyzwi56q0pfnmk061-nixos-system-gitlab-test.drv8 /nix/store/mx3q8741app2c5z43j58mh4p0k6zyijj-closure-info.drv9 /nix/store/7k210glcnbi2zl7853qlybbhhkp76li2-closure-info.drv10 /nix/store/0xjvaywijd9qrhjxmjgmpkb0yx47g9m8-run-nixos-vm.drv11 /nix/store/nm9p2hd7yw63wxmhqpsw81p9wrqayj5w-nixos-vm.drv12 /nix/store/407rq5hg4vf9cmlfbsal6llrvzd94l4k-driverConfiguration.json.drv13 /nix/store/hlh3p4ww8d17p0bpr89f86l6vzvgi845-nixos-test-driver-nixbot-gitlab.drv14 /nix/store/3q20xv8rdxb35vrqk7jz9adxzn2k6kcn-vm-test-run-nixbot-gitlab.drv15building '/nix/store/2vzniwmy0apswgmrcx4ajm0n8zh0hpjf-python3.13-nixbot.drv' on 'ssh-ng://nix@jamie'16building '/nix/store/2vzniwmy0apswgmrcx4ajm0n8zh0hpjf-python3.13-nixbot.drv'17python3.13-nixbot> Sourcing python-remove-tests-dir-hook18python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh19python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh20python3.13-nixbot> Sourcing pypa-build-hook21python3.13-nixbot> Using pypaBuildPhase22python3.13-nixbot> Sourcing python-runtime-deps-check-hook23python3.13-nixbot> Using pythonRuntimeDepsCheckHook24python3.13-nixbot> Sourcing pypa-install-hook25python3.13-nixbot> Using pypaInstallPhase26python3.13-nixbot> Sourcing python-imports-check-hook.sh27python3.13-nixbot> Using pythonImportsCheckPhase28python3.13-nixbot> Sourcing python-namespaces-hook29python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh30python3.13-nixbot> Running phase: unpackPhase31python3.13-nixbot> unpacking source archive /nix/store/k2ijhvd2xmdg11y7yn0n5yk7ifc7lsja-nixbot32python3.13-nixbot> source root is nixbot33python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml"34python3.13-nixbot> Running phase: patchPhase35python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase36python3.13-nixbot> Running phase: configurePhase37python3.13-nixbot> no configure script, doing nothing38python3.13-nixbot> Running phase: buildPhase39python3.13-nixbot> Executing pypaBuildPhase40python3.13-nixbot> Creating a wheel...41python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel42python3.13-nixbot> * Getting build dependencies for wheel...43python3.13-nixbot> * Building wheel...44python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl45python3.13-nixbot> Finished creating a wheel...46python3.13-nixbot> Finished executing pypaBuildPhase47python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook48python3.13-nixbot> Executing pythonRuntimeDepsCheck49python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl50python3.13-nixbot> Finished executing pythonRuntimeDepsCheck51python3.13-nixbot> Running phase: installPhase52python3.13-nixbot> Executing pypaInstallPhase53python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl54python3.13-nixbot> Finished executing pypaInstallPhase55python3.13-nixbot> Running phase: pythonOutputDistPhase56python3.13-nixbot> Executing pythonOutputDistPhase57python3.13-nixbot> Finished executing pythonOutputDistPhase58python3.13-nixbot> Running phase: fixupPhase59python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/6f9ljq8xb8kqlxmmqkybbhpv7z6f8b23-python3.13-nixbot60python3.13-nixbot> checking for references to /build/ in /nix/store/6f9ljq8xb8kqlxmmqkybbhpv7z6f8b23-python3.13-nixbot...61python3.13-nixbot> patching script interpreter paths in /nix/store/6f9ljq8xb8kqlxmmqkybbhpv7z6f8b23-python3.13-nixbot62python3.13-nixbot> stripping (with command strip and flags -S -p) in /nix/store/6f9ljq8xb8kqlxmmqkybbhpv7z6f8b23-python3.13-nixbot/lib /nix/store/6f9ljq8xb8kqlxmmqkybbhpv7z6f8b23-python3.13-nixbot/bin63python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/p2mg7pw1zgdsvbgfxm3lcp1218bmz5zd-python3.13-nixbot-dist64python3.13-nixbot> checking for references to /build/ in /nix/store/p2mg7pw1zgdsvbgfxm3lcp1218bmz5zd-python3.13-nixbot-dist...65python3.13-nixbot> patching script interpreter paths in /nix/store/p2mg7pw1zgdsvbgfxm3lcp1218bmz5zd-python3.13-nixbot-dist66python3.13-nixbot> Rewriting #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13 to #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.1367python3.13-nixbot> wrapping `/nix/store/6f9ljq8xb8kqlxmmqkybbhpv7z6f8b23-python3.13-nixbot/bin/nixbot'...68python3.13-nixbot> Executing pythonRemoveTestsDir69python3.13-nixbot> Finished executing pythonRemoveTestsDir70python3.13-nixbot> Running phase: pythonCatchConflictsPhase71python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase72python3.13-nixbot> Running phase: pythonImportsCheckPhase73python3.13-nixbot> Executing pythonImportsCheckPhase74building '/nix/store/dbhidf525lg7pm7g8iknm0xfdbl3jfwz-unit-nixbot.service.drv'75building '/nix/store/dw022wlrcp6hl42yxc794yhb7p1kqj52-system-units.drv'76building '/nix/store/81jyf6f1dpbrvx840wzx9gxig2b95zxw-etc.drv'77building '/nix/store/vfclw52j9lh2h4g59zfk8jlssmx5wv55-activate.drv'78building '/nix/store/q1iysgrk1dbzjq8hyzwi56q0pfnmk061-nixos-system-gitlab-test.drv'79nixos-system-gitlab-test> structuredAttrs is enabled80building '/nix/store/mx3q8741app2c5z43j58mh4p0k6zyijj-closure-info.drv'81closure-info> structuredAttrs is enabled82building '/nix/store/7k210glcnbi2zl7853qlybbhhkp76li2-closure-info.drv'83closure-info> structuredAttrs is enabled84building '/nix/store/0xjvaywijd9qrhjxmjgmpkb0yx47g9m8-run-nixos-vm.drv'85building '/nix/store/nm9p2hd7yw63wxmhqpsw81p9wrqayj5w-nixos-vm.drv'86building '/nix/store/407rq5hg4vf9cmlfbsal6llrvzd94l4k-driverConfiguration.json.drv'87driverConfiguration.json> structuredAttrs is enabled88building '/nix/store/hlh3p4ww8d17p0bpr89f86l6vzvgi845-nixos-test-driver-nixbot-gitlab.drv'89nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)90nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck91nixos-test-driver-nixbot-gitlab> All checks passed!92nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)93nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint94nixos-test-driver-nixbot-gitlab> All checks passed!95building '/nix/store/3q20xv8rdxb35vrqk7jz9adxzn2k6kcn-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie'96building '/nix/store/3q20xv8rdxb35vrqk7jz9adxzn2k6kcn-vm-test-run-nixbot-gitlab.drv'97vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state98vm-test-run-nixbot-gitlab> start all VLans99vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)100vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds101vm-test-run-nixbot-gitlab> run the VM test script102vm-test-run-nixbot-gitlab> additionally exposed symbols:103vm-test-run-nixbot-gitlab> gitlab,104vm-test-run-nixbot-gitlab> vlan1,105vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh106vm-test-run-nixbot-gitlab> start all VMs107vm-test-run-nixbot-gitlab> gitlab: starting vm108vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...109vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)110vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.Eaj4v16R27', fmt=raw size=1073741824111vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)112vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done113vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes114vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: f474be07-6860-49a3-91a9-9c438fa888c6115vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:116vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376117vm-test-run-nixbot-gitlab> (finished: start all VMs, in 35.11 seconds)118vm-test-run-nixbot-gitlab> gitlab # 119vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service120vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done121vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting122vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done123vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done124vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done125vm-test-run-nixbot-gitlab> gitlab # 126vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.127vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...128vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.129vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)130vm-test-run-nixbot-gitlab> gitlab # 131vm-test-run-nixbot-gitlab> gitlab # 132vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00133vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...134vm-test-run-nixbot-gitlab> gitlab # 135vm-test-run-nixbot-gitlab> gitlab # 136vm-test-run-nixbot-gitlab> gitlab # 137vm-test-run-nixbot-gitlab> gitlab # 138vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00139vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...140vm-test-run-nixbot-gitlab> gitlab # 141vm-test-run-nixbot-gitlab> gitlab # 142vm-test-run-nixbot-gitlab> gitlab # Booting from ROM...143vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok144vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026145vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3c8cfw70vhdlb9zswshrs4zpx9n4i5av-nixos-system-gitlab-test/init regInfo=/nix/store/3x4dsan0rjm4mykws0y8jnm30qvbz2c5-closure-info/registration console=ttyS0,115200n8 console=tty0146vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-provided physical RAM map:147vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable148vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved149vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved150vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable151vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved152vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved153vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved154vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable155vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved156vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NX (Execute Disable) protection: active157vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] APIC: Static calls initialized158vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SMBIOS 2.8 present.159vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014160vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: Memory slots populated: 1/1161vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Hypervisor detected: KVM162vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000163vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00164vm-test-run-nixbot-gitlab> gitlab # [ 0.000001] kvm-clock: using sched offset of 489381638 cycles165vm-test-run-nixbot-gitlab> gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns166vm-test-run-nixbot-gitlab> gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor167vm-test-run-nixbot-gitlab> gitlab # [ 0.000815] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000168vm-test-run-nixbot-gitlab> gitlab # [ 0.000852] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs169vm-test-run-nixbot-gitlab> gitlab # [ 0.000855] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT170vm-test-run-nixbot-gitlab> gitlab # [ 0.000906] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000171vm-test-run-nixbot-gitlab> gitlab # [ 0.002763] found SMP MP-table at [mem 0x000f5470-0x000f547f]172vm-test-run-nixbot-gitlab> gitlab # [ 0.002774] Using GB pages for direct mapping173vm-test-run-nixbot-gitlab> gitlab # [ 0.002889] RAMDISK: [mem 0xbe4e0000-0xbffcffff]174vm-test-run-nixbot-gitlab> gitlab # [ 0.002897] ACPI: Early table checksum verification disabled175vm-test-run-nixbot-gitlab> gitlab # [ 0.002899] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )176vm-test-run-nixbot-gitlab> gitlab # [ 0.002903] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)177vm-test-run-nixbot-gitlab> gitlab # [ 0.002907] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)178vm-test-run-nixbot-gitlab> gitlab # [ 0.002914] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)179vm-test-run-nixbot-gitlab> gitlab # [ 0.002916] ACPI: FACS 0x00000000BFFE0000 000040180vm-test-run-nixbot-gitlab> gitlab # [ 0.002917] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)181vm-test-run-nixbot-gitlab> gitlab # [ 0.002919] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)182vm-test-run-nixbot-gitlab> gitlab # [ 0.002920] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)183vm-test-run-nixbot-gitlab> gitlab # [ 0.002922] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]184vm-test-run-nixbot-gitlab> gitlab # [ 0.002923] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]185vm-test-run-nixbot-gitlab> gitlab # [ 0.002923] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]186vm-test-run-nixbot-gitlab> gitlab # [ 0.002924] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]187vm-test-run-nixbot-gitlab> gitlab # [ 0.002924] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]188vm-test-run-nixbot-gitlab> gitlab # [ 0.002925] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]189vm-test-run-nixbot-gitlab> gitlab # [ 0.003428] No NUMA configuration found190vm-test-run-nixbot-gitlab> gitlab # [ 0.003430] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]191vm-test-run-nixbot-gitlab> gitlab # [ 0.003433] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]192vm-test-run-nixbot-gitlab> gitlab # [ 0.005789] Zone ranges:193vm-test-run-nixbot-gitlab> gitlab # [ 0.005790] DMA [mem 0x0000000000001000-0x0000000000ffffff]194vm-test-run-nixbot-gitlab> gitlab # [ 0.005791] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]195vm-test-run-nixbot-gitlab> gitlab # [ 0.005793] Normal [mem 0x0000000100000000-0x00000001bfffffff]196vm-test-run-nixbot-gitlab> gitlab # [ 0.005794] Device empty197vm-test-run-nixbot-gitlab> gitlab # [ 0.005794] Movable zone start for each node198vm-test-run-nixbot-gitlab> gitlab # [ 0.005795] Early memory node ranges199vm-test-run-nixbot-gitlab> gitlab # [ 0.005796] node 0: [mem 0x0000000000001000-0x000000000009efff]200vm-test-run-nixbot-gitlab> gitlab # [ 0.005797] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]201vm-test-run-nixbot-gitlab> gitlab # [ 0.005798] node 0: [mem 0x0000000100000000-0x00000001bfffffff]202vm-test-run-nixbot-gitlab> gitlab # [ 0.005799] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]203vm-test-run-nixbot-gitlab> gitlab # [ 0.005819] On node 0, zone DMA: 1 pages in unavailable ranges204vm-test-run-nixbot-gitlab> gitlab # [ 0.006090] On node 0, zone DMA: 97 pages in unavailable ranges205vm-test-run-nixbot-gitlab> gitlab # [ 0.113926] On node 0, zone Normal: 40 pages in unavailable ranges206vm-test-run-nixbot-gitlab> gitlab # [ 0.114965] ACPI: PM-Timer IO Port: 0x608207vm-test-run-nixbot-gitlab> gitlab # [ 0.114982] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])208vm-test-run-nixbot-gitlab> gitlab # [ 0.115022] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23209vm-test-run-nixbot-gitlab> gitlab # [ 0.115024] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)210vm-test-run-nixbot-gitlab> gitlab # [ 0.115026] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)211vm-test-run-nixbot-gitlab> gitlab # [ 0.115027] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)212vm-test-run-nixbot-gitlab> gitlab # [ 0.115029] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)213vm-test-run-nixbot-gitlab> gitlab # [ 0.115029] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)214vm-test-run-nixbot-gitlab> gitlab # [ 0.115032] ACPI: Using ACPI (MADT) for SMP configuration information215vm-test-run-nixbot-gitlab> gitlab # [ 0.115033] ACPI: HPET id: 0x8086a201 base: 0xfed00000216vm-test-run-nixbot-gitlab> gitlab # [ 0.115040] TSC deadline timer available217vm-test-run-nixbot-gitlab> gitlab # [ 0.115044] CPU topo: Max. logical packages: 1218vm-test-run-nixbot-gitlab> gitlab # [ 0.115045] CPU topo: Max. logical dies: 1219vm-test-run-nixbot-gitlab> gitlab # [ 0.115045] CPU topo: Max. dies per package: 1220vm-test-run-nixbot-gitlab> gitlab # [ 0.115048] CPU topo: Max. threads per core: 1221vm-test-run-nixbot-gitlab> gitlab # [ 0.115049] CPU topo: Num. cores per package: 4222vm-test-run-nixbot-gitlab> gitlab # [ 0.115049] CPU topo: Num. threads per package: 4223vm-test-run-nixbot-gitlab> gitlab # [ 0.115050] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs224vm-test-run-nixbot-gitlab> gitlab # [ 0.115076] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()225vm-test-run-nixbot-gitlab> gitlab # [ 0.115095] kvm-guest: KVM setup pv remote TLB flush226vm-test-run-nixbot-gitlab> gitlab # [ 0.115098] kvm-guest: setup PV sched yield227vm-test-run-nixbot-gitlab> gitlab # [ 0.115120] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]228vm-test-run-nixbot-gitlab> gitlab # [ 0.115121] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]229vm-test-run-nixbot-gitlab> gitlab # [ 0.115122] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]230vm-test-run-nixbot-gitlab> gitlab # [ 0.115124] [mem 0xc0000000-0xfeffbfff] available for PCI devices231vm-test-run-nixbot-gitlab> gitlab # [ 0.115126] Booting paravirtualized kernel on KVM232vm-test-run-nixbot-gitlab> gitlab # [ 0.115129] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns233vm-test-run-nixbot-gitlab> gitlab # [ 0.119590] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1234vm-test-run-nixbot-gitlab> gitlab # [ 0.121931] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288235vm-test-run-nixbot-gitlab> gitlab # [ 0.121982] kvm-guest: PV spinlocks enabled236vm-test-run-nixbot-gitlab> gitlab # [ 0.121984] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)237vm-test-run-nixbot-gitlab> gitlab # [ 0.121987] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3c8cfw70vhdlb9zswshrs4zpx9n4i5av-nixos-system-gitlab-test/init regInfo=/nix/store/3x4dsan0rjm4mykws0y8jnm30qvbz2c5-closure-info/registration console=ttyS0,115200n8 console=tty0238vm-test-run-nixbot-gitlab> gitlab # [ 0.122083] Unknown kernel command line parameters "regInfo=/nix/store/3x4dsan0rjm4mykws0y8jnm30qvbz2c5-closure-info/registration", will be passed to user space.239vm-test-run-nixbot-gitlab> gitlab # [ 0.122096] random: crng init done240vm-test-run-nixbot-gitlab> gitlab # [ 0.122096] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes241vm-test-run-nixbot-gitlab> gitlab # [ 0.130748] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)242vm-test-run-nixbot-gitlab> gitlab # [ 0.134999] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)243vm-test-run-nixbot-gitlab> gitlab # [ 0.135073] software IO TLB: area num 4.244vm-test-run-nixbot-gitlab> gitlab # [ 0.207657] Fallback order for Node 0: 0245vm-test-run-nixbot-gitlab> gitlab # [ 0.207666] Built 1 zonelists, mobility grouping on. Total pages: 1572726246vm-test-run-nixbot-gitlab> gitlab # [ 0.207668] Policy zone: Normal247vm-test-run-nixbot-gitlab> gitlab # [ 0.210325] mem auto-init: stack:all(zero), heap alloc:on, heap free:off248vm-test-run-nixbot-gitlab> gitlab # [ 0.219729] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1249vm-test-run-nixbot-gitlab> gitlab # [ 0.232080] allocated 12582912 bytes of page_ext250vm-test-run-nixbot-gitlab> gitlab # [ 0.241771] ftrace: allocating 48565 entries in 192 pages251vm-test-run-nixbot-gitlab> gitlab # [ 0.241773] ftrace: allocated 192 pages with 2 groups252vm-test-run-nixbot-gitlab> gitlab # [ 0.242632] Dynamic Preempt: lazy253vm-test-run-nixbot-gitlab> gitlab # [ 0.242950] rcu: Preemptible hierarchical RCU implementation.254vm-test-run-nixbot-gitlab> gitlab # [ 0.242951] rcu: RCU event tracing is enabled.255vm-test-run-nixbot-gitlab> gitlab # [ 0.242951] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.256vm-test-run-nixbot-gitlab> gitlab # [ 0.242953] Trampoline variant of Tasks RCU enabled.257vm-test-run-nixbot-gitlab> gitlab # [ 0.242954] Rude variant of Tasks RCU enabled.258vm-test-run-nixbot-gitlab> gitlab # [ 0.242954] Tracing variant of Tasks RCU enabled.259vm-test-run-nixbot-gitlab> gitlab # [ 0.242955] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.260vm-test-run-nixbot-gitlab> gitlab # [ 0.242955] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4261vm-test-run-nixbot-gitlab> gitlab # [ 0.242975] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.262vm-test-run-nixbot-gitlab> gitlab # [ 0.242977] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.263vm-test-run-nixbot-gitlab> gitlab # [ 0.242978] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.264vm-test-run-nixbot-gitlab> gitlab # [ 0.247243] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16265vm-test-run-nixbot-gitlab> gitlab # [ 0.247525] rcu: srcu_init: Setting srcu_struct sizes based on contention.266vm-test-run-nixbot-gitlab> gitlab # [ 0.247637] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)267vm-test-run-nixbot-gitlab> gitlab # [ 0.254958] Console: colour VGA+ 80x25268vm-test-run-nixbot-gitlab> gitlab # [ 0.254962] printk: legacy console [tty0] enabled269vm-test-run-nixbot-gitlab> gitlab # [ 0.299044] printk: legacy console [ttyS0] enabled270vm-test-run-nixbot-gitlab> gitlab # [ 0.498598] ACPI: Core revision 20250807271vm-test-run-nixbot-gitlab> gitlab # [ 0.500245] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns272vm-test-run-nixbot-gitlab> gitlab # [ 0.503117] APIC: Switch to symmetric I/O mode setup273vm-test-run-nixbot-gitlab> gitlab # [ 0.504870] x2apic enabled274vm-test-run-nixbot-gitlab> gitlab # [ 0.506116] APIC: Switched APIC routing to: physical x2apic275vm-test-run-nixbot-gitlab> gitlab # [ 0.507825] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()276vm-test-run-nixbot-gitlab> gitlab # [ 0.510066] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()277vm-test-run-nixbot-gitlab> gitlab # [ 0.512835] kvm-guest: setup PV IPIs278vm-test-run-nixbot-gitlab> gitlab # [ 0.515215] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1279vm-test-run-nixbot-gitlab> gitlab # [ 0.517039] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns280vm-test-run-nixbot-gitlab> gitlab # [ 0.520212] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)281vm-test-run-nixbot-gitlab> gitlab # [ 0.521317] x86/cpu: User Mode Instruction Prevention (UMIP) activated282vm-test-run-nixbot-gitlab> gitlab # [ 0.523356] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127283vm-test-run-nixbot-gitlab> gitlab # [ 0.525199] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0284vm-test-run-nixbot-gitlab> gitlab # [ 0.527204] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto285vm-test-run-nixbot-gitlab> gitlab # [ 0.529201] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl286vm-test-run-nixbot-gitlab> gitlab # [ 0.531200] Transient Scheduler Attacks: Mitigation: Clear CPU buffers287vm-test-run-nixbot-gitlab> gitlab # [ 0.532199] Spectre V2 : Mitigation: Enhanced / Automatic IBRS288vm-test-run-nixbot-gitlab> gitlab # [ 0.533200] Speculative Return Stack Overflow: Mitigation: Safe RET289vm-test-run-nixbot-gitlab> gitlab # [ 0.535200] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization290vm-test-run-nixbot-gitlab> gitlab # [ 0.537205] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier291vm-test-run-nixbot-gitlab> gitlab # [ 0.539200] active return thunk: srso_alias_return_thunk292vm-test-run-nixbot-gitlab> gitlab # [ 0.540229] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'293vm-test-run-nixbot-gitlab> gitlab # [ 0.542200] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'294vm-test-run-nixbot-gitlab> gitlab # [ 0.544200] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'295vm-test-run-nixbot-gitlab> gitlab # [ 0.546200] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'296vm-test-run-nixbot-gitlab> gitlab # [ 0.548199] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'297vm-test-run-nixbot-gitlab> gitlab # [ 0.549200] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'298vm-test-run-nixbot-gitlab> gitlab # [ 0.550199] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'299vm-test-run-nixbot-gitlab> gitlab # [ 0.551200] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'300vm-test-run-nixbot-gitlab> gitlab # [ 0.553200] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'301vm-test-run-nixbot-gitlab> gitlab # [ 0.554200] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256302vm-test-run-nixbot-gitlab> gitlab # [ 0.556199] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64303vm-test-run-nixbot-gitlab> gitlab # [ 0.557200] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512304vm-test-run-nixbot-gitlab> gitlab # [ 0.558199] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024305vm-test-run-nixbot-gitlab> gitlab # [ 0.559200] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8306vm-test-run-nixbot-gitlab> gitlab # [ 0.560200] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16307vm-test-run-nixbot-gitlab> gitlab # [ 0.562200] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24308vm-test-run-nixbot-gitlab> gitlab # [ 0.564199] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.309vm-test-run-nixbot-gitlab> gitlab # [ 0.594608] Freeing SMP alternatives memory: 44K310vm-test-run-nixbot-gitlab> gitlab # [ 0.595202] pid_max: default: 32768 minimum: 301311vm-test-run-nixbot-gitlab> gitlab # [ 0.597320] LSM: initializing lsm=capability,landlock,yama,bpf,ima312vm-test-run-nixbot-gitlab> gitlab # [ 0.598318] landlock: Up and running.313vm-test-run-nixbot-gitlab> gitlab # [ 0.599200] Yama: becoming mindful.314vm-test-run-nixbot-gitlab> gitlab # [ 0.600410] LSM support for eBPF active315vm-test-run-nixbot-gitlab> gitlab # [ 0.602403] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)316vm-test-run-nixbot-gitlab> gitlab # [ 0.603346] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)317vm-test-run-nixbot-gitlab> gitlab # [ 0.606603] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)318vm-test-run-nixbot-gitlab> gitlab # [ 0.607840] Performance Events: Fam17h+ core perfctr, AMD PMU driver.319vm-test-run-nixbot-gitlab> gitlab # [ 0.608210] ... version: 2320vm-test-run-nixbot-gitlab> gitlab # [ 0.609201] ... bit width: 48321vm-test-run-nixbot-gitlab> gitlab # [ 0.610201] ... generic counters: 6322vm-test-run-nixbot-gitlab> gitlab # [ 0.611201] ... generic bitmap: 000000000000003f323vm-test-run-nixbot-gitlab> gitlab # [ 0.612201] ... fixed-purpose counters: 0324vm-test-run-nixbot-gitlab> gitlab # [ 0.613201] ... fixed-purpose bitmap: 0000000000000000325vm-test-run-nixbot-gitlab> gitlab # [ 0.614201] ... value mask: 0000ffffffffffff326vm-test-run-nixbot-gitlab> gitlab # [ 0.615201] ... max period: 00007fffffffffff327vm-test-run-nixbot-gitlab> gitlab # [ 0.616201] ... global_ctrl mask: 000000000000003f328vm-test-run-nixbot-gitlab> gitlab # [ 0.617325] signal: max sigframe size: 3376329vm-test-run-nixbot-gitlab> gitlab # [ 0.618286] rcu: Hierarchical SRCU implementation.330vm-test-run-nixbot-gitlab> gitlab # [ 0.619206] rcu: Max phase no-delay instances is 400.331vm-test-run-nixbot-gitlab> gitlab # [ 0.620402] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level332vm-test-run-nixbot-gitlab> gitlab # [ 0.625673] smp: Bringing up secondary CPUs ...333vm-test-run-nixbot-gitlab> gitlab # [ 0.626617] smpboot: x86: Booting SMP configuration:334vm-test-run-nixbot-gitlab> gitlab # [ 0.627212] .... node #0, CPUs: #1 #2 #3335vm-test-run-nixbot-gitlab> gitlab # [ 0.631367] smp: Brought up 1 node, 4 CPUs336vm-test-run-nixbot-gitlab> gitlab # [ 0.634205] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)337vm-test-run-nixbot-gitlab> gitlab # [ 0.635756] Memory: 6013176K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258292K reserved, 0K cma-reserved)338vm-test-run-nixbot-gitlab> gitlab # [ 0.638199] devtmpfs: initialized339vm-test-run-nixbot-gitlab> gitlab # [ 0.639399] x86/mm: Memory block size: 128MB340vm-test-run-nixbot-gitlab> gitlab # [ 0.642355] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns341vm-test-run-nixbot-gitlab> gitlab # [ 0.644252] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)342vm-test-run-nixbot-gitlab> gitlab # [ 0.645277] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).343vm-test-run-nixbot-gitlab> gitlab # [ 0.646329] pinctrl core: initialized pinctrl subsystem344vm-test-run-nixbot-gitlab> gitlab # [ 0.647575] PM: RTC time: 07:09:37, date: 2026-06-30345vm-test-run-nixbot-gitlab> gitlab # [ 0.651116] NET: Registered PF_NETLINK/PF_ROUTE protocol family346vm-test-run-nixbot-gitlab> gitlab # [ 0.653547] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations347vm-test-run-nixbot-gitlab> gitlab # [ 0.654266] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations348vm-test-run-nixbot-gitlab> gitlab # [ 0.656429] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations349vm-test-run-nixbot-gitlab> gitlab # [ 0.657213] audit: initializing netlink subsys (disabled)350vm-test-run-nixbot-gitlab> gitlab # [ 0.658365] audit: type=2000 audit(1782803377.024:1): state=initialized audit_enabled=0 res=1351vm-test-run-nixbot-gitlab> gitlab # [ 0.658518] thermal_sys: Registered thermal governor 'fair_share'352vm-test-run-nixbot-gitlab> gitlab # [ 0.660202] thermal_sys: Registered thermal governor 'bang_bang'353vm-test-run-nixbot-gitlab> gitlab # [ 0.661202] thermal_sys: Registered thermal governor 'step_wise'354vm-test-run-nixbot-gitlab> gitlab # [ 0.662202] thermal_sys: Registered thermal governor 'user_space'355vm-test-run-nixbot-gitlab> gitlab # [ 0.663202] thermal_sys: Registered thermal governor 'power_allocator'356vm-test-run-nixbot-gitlab> gitlab # [ 0.664217] cpuidle: using governor menu357vm-test-run-nixbot-gitlab> gitlab # [ 0.667443] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5358vm-test-run-nixbot-gitlab> gitlab # [ 0.668506] PCI: Using configuration type 1 for base access359vm-test-run-nixbot-gitlab> gitlab # [ 0.669202] PCI: Using configuration type 1 for extended access360vm-test-run-nixbot-gitlab> gitlab # [ 0.670383] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.361vm-test-run-nixbot-gitlab> gitlab # [ 0.676715] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages362vm-test-run-nixbot-gitlab> gitlab # [ 0.681212] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page363vm-test-run-nixbot-gitlab> gitlab # [ 0.686213] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages364vm-test-run-nixbot-gitlab> gitlab # [ 0.691209] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page365vm-test-run-nixbot-gitlab> gitlab # [ 0.701955] ACPI: Added _OSI(Module Device)366vm-test-run-nixbot-gitlab> gitlab # [ 0.705210] ACPI: Added _OSI(Processor Device)367vm-test-run-nixbot-gitlab> gitlab # [ 0.708210] ACPI: Added _OSI(Processor Aggregator Device)368vm-test-run-nixbot-gitlab> gitlab # [ 0.717253] ACPI: 1 ACPI AML tables successfully acquired and loaded369vm-test-run-nixbot-gitlab> gitlab # [ 0.725985] ACPI: Interpreter enabled370vm-test-run-nixbot-gitlab> gitlab # [ 0.729350] ACPI: PM: (supports S0 S3 S4 S5)371vm-test-run-nixbot-gitlab> gitlab # [ 0.732212] ACPI: Using IOAPIC for interrupt routing372vm-test-run-nixbot-gitlab> gitlab # [ 0.736516] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug373vm-test-run-nixbot-gitlab> gitlab # [ 0.743210] PCI: Using E820 reservations for host bridge windows374vm-test-run-nixbot-gitlab> gitlab # [ 0.747798] ACPI: Enabled 2 GPEs in block 00 to 0F375vm-test-run-nixbot-gitlab> gitlab # [ 0.764370] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])376vm-test-run-nixbot-gitlab> gitlab # [ 0.767235] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]377vm-test-run-nixbot-gitlab> gitlab # [ 0.774276] acpiphp: Slot [3] registered378vm-test-run-nixbot-gitlab> gitlab # [ 0.776292] acpiphp: Slot [4] registered379vm-test-run-nixbot-gitlab> gitlab # [ 0.779291] acpiphp: Slot [5] registered380vm-test-run-nixbot-gitlab> gitlab # [ 0.781341] acpiphp: Slot [6] registered381vm-test-run-nixbot-gitlab> gitlab # [ 0.783245] acpiphp: Slot [7] registered382vm-test-run-nixbot-gitlab> gitlab # [ 0.785245] acpiphp: Slot [8] registered383vm-test-run-nixbot-gitlab> gitlab # [ 0.786245] acpiphp: Slot [9] registered384vm-test-run-nixbot-gitlab> gitlab # [ 0.787266] acpiphp: Slot [10] registered385vm-test-run-nixbot-gitlab> gitlab # [ 0.788252] acpiphp: Slot [11] registered386vm-test-run-nixbot-gitlab> gitlab # [ 0.790245] acpiphp: Slot [12] registered387vm-test-run-nixbot-gitlab> gitlab # [ 0.791260] acpiphp: Slot [13] registered388vm-test-run-nixbot-gitlab> gitlab # [ 0.792236] acpiphp: Slot [14] registered389vm-test-run-nixbot-gitlab> gitlab # [ 0.794531] acpiphp: Slot [15] registered390vm-test-run-nixbot-gitlab> gitlab # [ 0.795238] acpiphp: Slot [16] registered391vm-test-run-nixbot-gitlab> gitlab # [ 0.798294] acpiphp: Slot [17] registered392vm-test-run-nixbot-gitlab> gitlab # [ 0.802323] acpiphp: Slot [18] registered393vm-test-run-nixbot-gitlab> gitlab # [ 0.805292] acpiphp: Slot [19] registered394vm-test-run-nixbot-gitlab> gitlab # [ 0.808310] acpiphp: Slot [20] registered395vm-test-run-nixbot-gitlab> gitlab # [ 0.811290] acpiphp: Slot [21] registered396vm-test-run-nixbot-gitlab> gitlab # [ 0.814387] acpiphp: Slot [22] registered397vm-test-run-nixbot-gitlab> gitlab # [ 0.818295] acpiphp: Slot [23] registered398vm-test-run-nixbot-gitlab> gitlab # [ 0.821291] acpiphp: Slot [24] registered399vm-test-run-nixbot-gitlab> gitlab # [ 0.824293] acpiphp: Slot [25] registered400vm-test-run-nixbot-gitlab> gitlab # [ 0.827326] acpiphp: Slot [26] registered401vm-test-run-nixbot-gitlab> gitlab # [ 0.831293] acpiphp: Slot [27] registered402vm-test-run-nixbot-gitlab> gitlab # [ 0.834291] acpiphp: Slot [28] registered403vm-test-run-nixbot-gitlab> gitlab # [ 0.837293] acpiphp: Slot [29] registered404vm-test-run-nixbot-gitlab> gitlab # [ 0.840299] acpiphp: Slot [30] registered405vm-test-run-nixbot-gitlab> gitlab # [ 0.843273] acpiphp: Slot [31] registered406vm-test-run-nixbot-gitlab> gitlab # [ 0.845348] PCI host bridge to bus 0000:00407vm-test-run-nixbot-gitlab> gitlab # [ 0.848248] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]408vm-test-run-nixbot-gitlab> gitlab # [ 0.852209] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]409vm-test-run-nixbot-gitlab> gitlab # [ 0.856216] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]410vm-test-run-nixbot-gitlab> gitlab # [ 0.859203] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]411vm-test-run-nixbot-gitlab> gitlab # [ 0.861215] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]412vm-test-run-nixbot-gitlab> gitlab # [ 0.863210] pci_bus 0000:00: root bus resource [bus 00-ff]413vm-test-run-nixbot-gitlab> gitlab # [ 0.867408] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint414vm-test-run-nixbot-gitlab> gitlab # [ 0.875610] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint415vm-test-run-nixbot-gitlab> gitlab # [ 0.884519] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint416vm-test-run-nixbot-gitlab> gitlab # [ 0.895339] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]417vm-test-run-nixbot-gitlab> gitlab # [ 0.898340] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk418vm-test-run-nixbot-gitlab> gitlab # [ 0.902208] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk419vm-test-run-nixbot-gitlab> gitlab # [ 0.906209] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk420vm-test-run-nixbot-gitlab> gitlab # [ 0.910209] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk421vm-test-run-nixbot-gitlab> gitlab # [ 0.913650] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint422vm-test-run-nixbot-gitlab> gitlab # [ 0.917280] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]423vm-test-run-nixbot-gitlab> gitlab # [ 0.920539] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint424vm-test-run-nixbot-gitlab> gitlab # [ 0.923940] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI425vm-test-run-nixbot-gitlab> gitlab # [ 0.926218] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB426vm-test-run-nixbot-gitlab> gitlab # [ 0.929570] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint427vm-test-run-nixbot-gitlab> gitlab # [ 0.952382] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]428vm-test-run-nixbot-gitlab> gitlab # [ 0.957279] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]429vm-test-run-nixbot-gitlab> gitlab # [ 0.962343] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]430vm-test-run-nixbot-gitlab> gitlab # [ 0.967770] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]431vm-test-run-nixbot-gitlab> gitlab # [ 0.999313] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 31250 usecs432vm-test-run-nixbot-gitlab> gitlab # [ 1.002519] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint433vm-test-run-nixbot-gitlab> gitlab # [ 1.009238] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]434vm-test-run-nixbot-gitlab> gitlab # [ 1.010217] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]435vm-test-run-nixbot-gitlab> gitlab # [ 1.012265] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]436vm-test-run-nixbot-gitlab> gitlab # [ 1.014218] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]437vm-test-run-nixbot-gitlab> gitlab # [ 1.018518] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint438vm-test-run-nixbot-gitlab> gitlab # [ 1.024240] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]439vm-test-run-nixbot-gitlab> gitlab # [ 1.026227] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]440vm-test-run-nixbot-gitlab> gitlab # [ 1.028257] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]441vm-test-run-nixbot-gitlab> gitlab # [ 1.032479] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint442vm-test-run-nixbot-gitlab> gitlab # [ 1.037818] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]443vm-test-run-nixbot-gitlab> gitlab # [ 1.039217] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]444vm-test-run-nixbot-gitlab> gitlab # [ 1.041256] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]445vm-test-run-nixbot-gitlab> gitlab # [ 1.045328] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint446vm-test-run-nixbot-gitlab> gitlab # [ 1.051217] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]447vm-test-run-nixbot-gitlab> gitlab # [ 1.053217] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]448vm-test-run-nixbot-gitlab> gitlab # [ 1.054255] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]449vm-test-run-nixbot-gitlab> gitlab # [ 1.059255] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint450vm-test-run-nixbot-gitlab> gitlab # [ 1.064263] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]451vm-test-run-nixbot-gitlab> gitlab # [ 1.066217] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]452vm-test-run-nixbot-gitlab> gitlab # [ 1.068361] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]453vm-test-run-nixbot-gitlab> gitlab # [ 1.080564] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint454vm-test-run-nixbot-gitlab> gitlab # [ 1.096246] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]455vm-test-run-nixbot-gitlab> gitlab # [ 1.101248] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]456vm-test-run-nixbot-gitlab> gitlab # [ 1.105343] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]457vm-test-run-nixbot-gitlab> gitlab # [ 1.116207] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint458vm-test-run-nixbot-gitlab> gitlab # [ 1.127242] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]459vm-test-run-nixbot-gitlab> gitlab # [ 1.130257] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]460vm-test-run-nixbot-gitlab> gitlab # [ 1.133257] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]461vm-test-run-nixbot-gitlab> gitlab # [ 1.135219] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]462vm-test-run-nixbot-gitlab> gitlab # [ 1.139482] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint463vm-test-run-nixbot-gitlab> gitlab # [ 1.144245] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]464vm-test-run-nixbot-gitlab> gitlab # [ 1.146256] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]465vm-test-run-nixbot-gitlab> gitlab # [ 1.150242] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint466vm-test-run-nixbot-gitlab> gitlab # [ 1.156247] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]467vm-test-run-nixbot-gitlab> gitlab # [ 1.157217] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]468vm-test-run-nixbot-gitlab> gitlab # [ 1.159255] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]469vm-test-run-nixbot-gitlab> gitlab # [ 1.163433] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint470vm-test-run-nixbot-gitlab> gitlab # [ 1.168227] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]471vm-test-run-nixbot-gitlab> gitlab # [ 1.170217] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]472vm-test-run-nixbot-gitlab> gitlab # [ 1.172258] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]473vm-test-run-nixbot-gitlab> gitlab # [ 1.181613] ACPI: PCI: Interrupt link LNKA configured for IRQ 10474vm-test-run-nixbot-gitlab> gitlab # [ 1.183422] ACPI: PCI: Interrupt link LNKB configured for IRQ 10475vm-test-run-nixbot-gitlab> gitlab # [ 1.186415] ACPI: PCI: Interrupt link LNKC configured for IRQ 11476vm-test-run-nixbot-gitlab> gitlab # [ 1.188431] ACPI: PCI: Interrupt link LNKD configured for IRQ 11477vm-test-run-nixbot-gitlab> gitlab # [ 1.190315] ACPI: PCI: Interrupt link LNKS configured for IRQ 9478vm-test-run-nixbot-gitlab> gitlab # [ 1.194253] iommu: Default domain type: Translated479vm-test-run-nixbot-gitlab> gitlab # [ 1.195215] iommu: DMA domain TLB invalidation policy: lazy mode480vm-test-run-nixbot-gitlab> gitlab # [ 1.200828] ACPI: bus type USB registered481vm-test-run-nixbot-gitlab> gitlab # [ 1.204496] usbcore: registered new interface driver usbfs482vm-test-run-nixbot-gitlab> gitlab # [ 1.209451] usbcore: registered new interface driver hub483vm-test-run-nixbot-gitlab> gitlab # [ 1.214258] usbcore: registered new device driver usb484vm-test-run-nixbot-gitlab> gitlab # [ 1.221595] NetLabel: Initializing485vm-test-run-nixbot-gitlab> gitlab # [ 1.224210] NetLabel: domain hash size = 128486vm-test-run-nixbot-gitlab> gitlab # [ 1.228210] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO487vm-test-run-nixbot-gitlab> gitlab # [ 1.232576] NetLabel: unlabeled traffic allowed by default488vm-test-run-nixbot-gitlab> gitlab # [ 1.237247] PCI: Using ACPI for IRQ routing489vm-test-run-nixbot-gitlab> gitlab # [ 1.243292] pci 0000:00:02.0: vgaarb: setting as boot VGA device490vm-test-run-nixbot-gitlab> gitlab # [ 1.244197] pci 0000:00:02.0: vgaarb: bridge control possible491vm-test-run-nixbot-gitlab> gitlab # [ 1.244197] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none492vm-test-run-nixbot-gitlab> gitlab # [ 1.259218] vgaarb: loaded493vm-test-run-nixbot-gitlab> gitlab # [ 1.261490] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0494vm-test-run-nixbot-gitlab> gitlab # [ 1.262204] hpet0: 3 comparators, 64-bit 100.000000 MHz counter495vm-test-run-nixbot-gitlab> gitlab # [ 1.267499] clocksource: Switched to clocksource kvm-clock496vm-test-run-nixbot-gitlab> gitlab # [ 1.270942] VFS: Disk quotas dquot_6.6.0497vm-test-run-nixbot-gitlab> gitlab # [ 1.272258] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)498vm-test-run-nixbot-gitlab> gitlab # [ 1.275210] pnp: PnP ACPI init499vm-test-run-nixbot-gitlab> gitlab # [ 1.277035] pnp: PnP ACPI: found 6 devices500vm-test-run-nixbot-gitlab> gitlab # [ 1.288386] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns501vm-test-run-nixbot-gitlab> gitlab # [ 1.291088] clocksource: Switched to clocksource acpi_pm502vm-test-run-nixbot-gitlab> gitlab # [ 1.292911] NET: Registered PF_INET protocol family503vm-test-run-nixbot-gitlab> gitlab # [ 1.295813] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)504vm-test-run-nixbot-gitlab> gitlab # [ 1.316074] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)505vm-test-run-nixbot-gitlab> gitlab # [ 1.318719] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)506vm-test-run-nixbot-gitlab> gitlab # [ 1.321121] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)507vm-test-run-nixbot-gitlab> gitlab # [ 1.325988] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)508vm-test-run-nixbot-gitlab> gitlab # [ 1.328343] TCP: Hash tables configured (established 65536 bind 65536)509vm-test-run-nixbot-gitlab> gitlab # [ 1.330450] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)510vm-test-run-nixbot-gitlab> gitlab # [ 1.332820] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)511vm-test-run-nixbot-gitlab> gitlab # [ 1.335188] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)512vm-test-run-nixbot-gitlab> gitlab # [ 1.337541] NET: Registered PF_UNIX/PF_LOCAL protocol family513vm-test-run-nixbot-gitlab> gitlab # [ 1.339397] NET: Registered PF_XDP protocol family514vm-test-run-nixbot-gitlab> gitlab # [ 1.340966] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]515vm-test-run-nixbot-gitlab> gitlab # [ 1.342836] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]516vm-test-run-nixbot-gitlab> gitlab # [ 1.344718] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]517vm-test-run-nixbot-gitlab> gitlab # [ 1.346759] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]518vm-test-run-nixbot-gitlab> gitlab # [ 1.348796] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]519vm-test-run-nixbot-gitlab> gitlab # [ 1.351113] pci 0000:00:01.0: PIIX3: Enabling Passive Release520vm-test-run-nixbot-gitlab> gitlab # [ 1.352882] pci 0000:00:00.0: Limiting direct PCI/PCI transfers521vm-test-run-nixbot-gitlab> gitlab # [ 1.356352] ACPI: \_SB_.LNKD: Enabled at IRQ 11522vm-test-run-nixbot-gitlab> gitlab # [ 1.359794] PCI: CLS 0 bytes, default 64523vm-test-run-nixbot-gitlab> gitlab # [ 1.361238] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)524vm-test-run-nixbot-gitlab> gitlab # [ 1.361952] Trying to unpack rootfs image as initramfs...525vm-test-run-nixbot-gitlab> gitlab # [ 1.363191] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)526vm-test-run-nixbot-gitlab> gitlab # [ 1.363333] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns527vm-test-run-nixbot-gitlab> gitlab # [ 1.390526] Initialise system trusted keyrings528vm-test-run-nixbot-gitlab> gitlab # [ 1.392624] workingset: timestamp_bits=40 max_order=21 bucket_order=0529vm-test-run-nixbot-gitlab> gitlab # [ 1.425364] Key type asymmetric registered530vm-test-run-nixbot-gitlab> gitlab # [ 1.428866] Asymmetric key parser 'x509' registered531vm-test-run-nixbot-gitlab> gitlab # [ 1.433021] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)532vm-test-run-nixbot-gitlab> gitlab # [ 1.439317] io scheduler mq-deadline registered533vm-test-run-nixbot-gitlab> gitlab # [ 1.441810] io scheduler kyber registered534vm-test-run-nixbot-gitlab> gitlab # [ 1.446575] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled535vm-test-run-nixbot-gitlab> gitlab # [ 1.448967] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A536vm-test-run-nixbot-gitlab> gitlab # [ 1.453222] Linux agpgart interface v0.103537vm-test-run-nixbot-gitlab> gitlab # [ 1.454611] ACPI: bus type drm_connector registered538vm-test-run-nixbot-gitlab> gitlab # [ 1.458831] usbcore: registered new interface driver usbserial_generic539vm-test-run-nixbot-gitlab> gitlab # [ 1.460784] usbserial: USB Serial support registered for generic540vm-test-run-nixbot-gitlab> gitlab # [ 1.462609] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.541vm-test-run-nixbot-gitlab> gitlab # [ 1.462609] Please enable it if your BIOS has the CPPC option.542vm-test-run-nixbot-gitlab> gitlab # [ 1.466619] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled543vm-test-run-nixbot-gitlab> gitlab # [ 1.469056] drop_monitor: Initializing network drop monitor service544vm-test-run-nixbot-gitlab> gitlab # [ 1.471099] NET: Registered PF_INET6 protocol family545vm-test-run-nixbot-gitlab> gitlab # [ 1.473722] Segment Routing with IPv6546vm-test-run-nixbot-gitlab> gitlab # [ 1.474946] In-situ OAM (IOAM) with IPv6547vm-test-run-nixbot-gitlab> gitlab # [ 1.477868] IPI shorthand broadcast: enabled548vm-test-run-nixbot-gitlab> gitlab # [ 1.483131] sched_clock: Marking stable (1222030885, 260663340)->(1556770305, -74076080)549vm-test-run-nixbot-gitlab> gitlab # [ 1.486188] registered taskstats version 1550vm-test-run-nixbot-gitlab> gitlab # [ 1.487963] Loading compiled-in X.509 certificates551vm-test-run-nixbot-gitlab> gitlab # [ 1.498579] Demotion targets for Node 0: null552vm-test-run-nixbot-gitlab> gitlab # [ 1.500378] Key type .fscrypt registered553vm-test-run-nixbot-gitlab> gitlab # [ 1.501659] Key type fscrypt-provisioning registered554vm-test-run-nixbot-gitlab> gitlab # [ 1.503475] ima: No TPM chip found, activating TPM-bypass!555vm-test-run-nixbot-gitlab> gitlab # [ 1.505230] ima: Allocated hash algorithm: sha1556vm-test-run-nixbot-gitlab> gitlab # [ 1.506741] ima: No architecture policies found557vm-test-run-nixbot-gitlab> gitlab # [ 1.508549] PM: Magic number: 14:611:168558vm-test-run-nixbot-gitlab> gitlab # [ 1.510962] RAS: Correctable Errors collector initialized.559vm-test-run-nixbot-gitlab> gitlab # [ 1.516499] clk: Disabling unused clocks560vm-test-run-nixbot-gitlab> gitlab # [ 1.517825] PM: genpd: Disabling unused power domains561vm-test-run-nixbot-gitlab> gitlab # [ 1.666358] Freeing initrd memory: 27584K562vm-test-run-nixbot-gitlab> gitlab # [ 1.675003] Freeing unused decrypted memory: 2028K563vm-test-run-nixbot-gitlab> gitlab # [ 1.683591] Freeing unused kernel image (initmem) memory: 3644K564vm-test-run-nixbot-gitlab> gitlab # [ 1.688416] Write protecting the kernel read-only data: 32768k565vm-test-run-nixbot-gitlab> gitlab # [ 1.695982] Freeing unused kernel image (text/rodata gap) memory: 1332K566vm-test-run-nixbot-gitlab> gitlab # [ 1.702485] Freeing unused kernel image (rodata/data gap) memory: 816K567vm-test-run-nixbot-gitlab> gitlab # [ 1.767903] x86/mm: Checked W+X mappings: passed, no W+X pages found.568vm-test-run-nixbot-gitlab> gitlab # [ 1.769896] Run /init as init process569vm-test-run-nixbot-gitlab> gitlab # [ 1.784379] systemd[1]: Inserted module 'autofs4'570vm-test-run-nixbot-gitlab> gitlab # [ 1.811866] fuse: init (API version 7.45)571vm-test-run-nixbot-gitlab> gitlab # [ 1.822495] ACPI: \_SB_.LNKC: Enabled at IRQ 10572vm-test-run-nixbot-gitlab> gitlab # [ 1.833396] ACPI: \_SB_.LNKA: Enabled at IRQ 10573vm-test-run-nixbot-gitlab> gitlab # [ 1.838368] ACPI: \_SB_.LNKB: Enabled at IRQ 11574vm-test-run-nixbot-gitlab> gitlab # [ 1.908609] systemd[1]: Successfully made /usr/ read-only.575vm-test-run-nixbot-gitlab> gitlab # [ 2.246295] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)576vm-test-run-nixbot-gitlab> gitlab # [ 2.263315] systemd[1]: Detected virtualization kvm.577vm-test-run-nixbot-gitlab> gitlab # [ 2.267293] systemd[1]: Detected architecture x86-64.578vm-test-run-nixbot-gitlab> gitlab # [ 2.271361] systemd[1]: Running in initrd.579vm-test-run-nixbot-gitlab> gitlab # [ 2.275799] systemd[1]: Initializing machine ID from random generator.580vm-test-run-nixbot-gitlab> gitlab # [ 2.281113] systemd[1]: Hostname set to <gitlab>.581vm-test-run-nixbot-gitlab> gitlab # [ 2.375296] systemd[1]: Queued start job for default target Initrd Default Target.582vm-test-run-nixbot-gitlab> gitlab # [ 2.551786] systemd[1]: Created slice Slice /system/modprobe.583vm-test-run-nixbot-gitlab> gitlab # [ 2.553929] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.584vm-test-run-nixbot-gitlab> gitlab # [ 2.556419] systemd[1]: Expecting device /dev/disk/by-label/nix-store...585vm-test-run-nixbot-gitlab> gitlab # [ 2.558544] systemd[1]: Expecting device /dev/disk/by-label/nixos...586vm-test-run-nixbot-gitlab> gitlab # [ 2.560599] systemd[1]: Reached target Path Units.587vm-test-run-nixbot-gitlab> gitlab # [ 2.562255] systemd[1]: Reached target Slice Units.588vm-test-run-nixbot-gitlab> gitlab # [ 2.563869] systemd[1]: Reached target Swaps.589vm-test-run-nixbot-gitlab> gitlab # [ 2.565390] systemd[1]: Reached target Timer Units.590vm-test-run-nixbot-gitlab> gitlab # [ 2.567189] systemd[1]: Listening on D-Bus System Message Bus Socket.591vm-test-run-nixbot-gitlab> gitlab # [ 2.569421] systemd[1]: Listening on Journal Socket (/dev/log).592vm-test-run-nixbot-gitlab> gitlab # [ 2.571540] systemd[1]: Listening on Journal Sockets.593vm-test-run-nixbot-gitlab> gitlab # [ 2.573401] systemd[1]: Listening on udev Control Socket.594vm-test-run-nixbot-gitlab> gitlab # [ 2.575319] systemd[1]: Listening on udev Kernel Socket.595vm-test-run-nixbot-gitlab> gitlab # [ 2.577074] systemd[1]: Reached target Socket Units.596vm-test-run-nixbot-gitlab> gitlab # [ 2.583135] systemd[1]: Starting Create List of Static Device Nodes...597vm-test-run-nixbot-gitlab> gitlab # [ 2.589374] systemd[1]: Starting Load Kernel Module 9pnet_virtio...598vm-test-run-nixbot-gitlab> gitlab # [ 2.609354] systemd[1]: Starting Load Kernel Module configfs...599vm-test-run-nixbot-gitlab> gitlab # [ 2.614880] systemd[1]: Starting Journal Service...600vm-test-run-nixbot-gitlab> gitlab # [ 2.623030] systemd[1]: Starting Load Kernel Modules...601vm-test-run-nixbot-gitlab> gitlab # [ 2.627462] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki602vm-test-run-nixbot-gitlab> gitlab # [ 2.640039] systemd-journald[151]: Collecting audit messages is disabled.603vm-test-run-nixbot-gitlab> gitlab # [ 2.685159] systemd[1]: Starting Coldplug All udev Devices...604vm-test-run-nixbot-gitlab> gitlab # [ 2.687413] netfs: FS-Cache loaded605vm-test-run-nixbot-gitlab> gitlab # [ 2.695099] 9pnet: Installing 9P2000 support606vm-test-run-nixbot-gitlab> gitlab # [ 2.702724] systemd[1]: Finished Create List of Static Device Nodes.607vm-test-run-nixbot-gitlab> gitlab # [ 2.706015] systemd[1]: modprobe@configfs.service: Deactivated successfully.608vm-test-run-nixbot-gitlab> gitlab # [ 2.710853] systemd[1]: Finished Load Kernel Module configfs.609vm-test-run-nixbot-gitlab> gitlab # [ 2.716442] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.610vm-test-run-nixbot-gitlab> gitlab # [ 2.718809] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.611vm-test-run-nixbot-gitlab> gitlab # [ 2.458258] systemd-modules-load[153]: Using 4 probe threads[ 2.720892] systemd[1]: Finished Load Kernel Module 9pnet_virtio.612vm-test-run-nixbot-gitlab> gitlab # [ 2.722545] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev613vm-test-run-nixbot-gitlab> gitlab # 614vm-test-run-nixbot-gitlab> gitlab # [ 2.466485] systemd-modules-load[153]: Inserted module 'virtio_balloon'615vm-test-run-nixbot-gitlab> gitlab # [ 2.728952] systemd[1]: Started Journal Service.616vm-test-run-nixbot-gitlab> gitlab # [ 2.468688] systemd-modules-load[153]: Inserted module 'virtio_gpu'617vm-test-run-nixbot-gitlab> gitlab # [ 2.471626] systemd-modules-load[153]: Inserted module 'dm_mod'618vm-test-run-nixbot-gitlab> gitlab # [ 2.473663] systemd[1]: Finished Load Kernel Modules.619vm-test-run-nixbot-gitlab> gitlab # [ 2.475594] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config620vm-test-run-nixbot-gitlab> gitlab # [ 2.478717] systemd[1]: Starting Apply Kernel Variables...621vm-test-run-nixbot-gitlab> gitlab # [ 2.490451] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...622vm-test-run-nixbot-gitlab> gitlab # [ 2.498375] systemd[1]: Finished Apply Kernel Variables.623vm-test-run-nixbot-gitlab> gitlab # [ 2.503728] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.624vm-test-run-nixbot-gitlab> gitlab # [ 2.508182] systemd[1]: Starting Create Static Device Nodes in /dev...625vm-test-run-nixbot-gitlab> gitlab # [ 2.528348] systemd[1]: Finished Create Static Device Nodes in /dev.626vm-test-run-nixbot-gitlab> gitlab # [ 2.530154] systemd[1]: Reached target Preparation for Local File Systems.627vm-test-run-nixbot-gitlab> gitlab # [ 2.531872] systemd[1]: Reached target Local File Systems.628vm-test-run-nixbot-gitlab> gitlab # [ 2.533397] systemd[1]: Starting Create System Files and Directories...629vm-test-run-nixbot-gitlab> gitlab # [ 2.536882] systemd[1]: Starting Rule-based Manager for Device Events and Files...630vm-test-run-nixbot-gitlab> gitlab # [ 2.547864] systemd[1]: Finished Create System Files and Directories.631vm-test-run-nixbot-gitlab> gitlab # [ 2.555711] systemd-udevd[192]: Using default interface naming scheme 'v260'.632vm-test-run-nixbot-gitlab> gitlab # [ 2.571919] systemd[1]: Started Rule-based Manager for Device Events and Files.633vm-test-run-nixbot-gitlab> gitlab # [ 2.611172] systemd[1]: Finished Coldplug All udev Devices.634vm-test-run-nixbot-gitlab> gitlab # [ 2.613751] systemd[1]: Reached target System Initialization.635vm-test-run-nixbot-gitlab> gitlab # [ 2.616329] systemd[1]: Reached target Basic System.636vm-test-run-nixbot-gitlab> gitlab # [ 2.998178] SCSI subsystem initialized637vm-test-run-nixbot-gitlab> gitlab # [ 3.032069] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12638vm-test-run-nixbot-gitlab> gitlab # [ 3.036758] virtio_blk virtio4: 4/0/0 default/read/poll queues639vm-test-run-nixbot-gitlab> gitlab # [ 3.038747] serio: i8042 KBD port at 0x60,0x64 irq 1640vm-test-run-nixbot-gitlab> gitlab # [ 3.038759] serio: i8042 AUX port at 0x60,0x64 irq 12641vm-test-run-nixbot-gitlab> gitlab # [ 3.041761] scsi host0: ata_piix642vm-test-run-nixbot-gitlab> gitlab # [ 3.044769] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)643vm-test-run-nixbot-gitlab> gitlab # [ 3.046276] uhci_hcd 0000:00:01.2: UHCI Host Controller644vm-test-run-nixbot-gitlab> gitlab # [ 3.049736] scsi host1: ata_piix645vm-test-run-nixbot-gitlab> gitlab # [ 3.050116] virtio_blk virtio5: 4/0/0 default/read/poll queues646vm-test-run-nixbot-gitlab> gitlab # [ 3.050424] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0647vm-test-run-nixbot-gitlab> gitlab # [ 3.051162] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1648vm-test-run-nixbot-gitlab> gitlab # [ 3.052037] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0649vm-test-run-nixbot-gitlab> gitlab # [ 3.061716] virtio_blk virtio5: [vdb] 11498128 512-byte logical blocks (5.89 GB/5.48 GiB)650vm-test-run-nixbot-gitlab> gitlab # [ 3.064779] uhci_hcd 0000:00:01.2: detected 2 ports651vm-test-run-nixbot-gitlab> gitlab # [ 3.066423] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140652vm-test-run-nixbot-gitlab> gitlab # [ 3.068145] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18653vm-test-run-nixbot-gitlab> gitlab # [ 3.070190] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1654vm-test-run-nixbot-gitlab> gitlab # [ 3.071968] usb usb1: Product: UHCI Host Controller655vm-test-run-nixbot-gitlab> gitlab # [ 3.073252] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd656vm-test-run-nixbot-gitlab> gitlab # [ 3.074629] usb usb1: SerialNumber: 0000:00:01.2657vm-test-run-nixbot-gitlab> gitlab # [ 3.076162] hub 1-0:1.0: USB hub found658vm-test-run-nixbot-gitlab> gitlab # [ 3.077203] hub 1-0:1.0: 2 ports detected659vm-test-run-nixbot-gitlab> gitlab # [ 2.829975] systemd[1]: Starting Virtual Console Setup...660vm-test-run-nixbot-gitlab> gitlab # [ 2.862601] systemd-vconsole-setup[231]: Configuration of first virtual console was skipped, ignoring remaining ones.661vm-test-run-nixbot-gitlab> gitlab # [ 2.867774] systemd[1]: Finished Virtual Console Setup.662vm-test-run-nixbot-gitlab> gitlab # [ 3.212255] ata2: found unknown device (class 0)663vm-test-run-nixbot-gitlab> gitlab # [ 3.216769] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100664vm-test-run-nixbot-gitlab> gitlab # [ 3.223768] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5665vm-test-run-nixbot-gitlab> gitlab # [ 3.015135] (udev-worker)[208[ 3.275250] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0666vm-test-run-nixbot-gitlab> gitlab # ]: Network interface NamePolicy= disabled on kernel command line.667vm-test-run-nixbot-gitlab> gitlab # [ 3.030344] (udev-worker)[221]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.668vm-test-run-nixbot-gitlab> gitlab # [ 3.040416] ([ 3.300774] usb 1-1: new full-speed USB device number 2 using uhci_hcd669vm-test-run-nixbot-gitlab> gitlab # udev-worker)[221]: Network interface NamePolicy= disabled on kernel command line.670vm-test-run-nixbot-gitlab> gitlab # [ 3.314847] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray671vm-test-run-nixbot-gitlab> gitlab # [ 3.054964] systemd[1]: Found device /dev/disk/by-label/nixos.672vm-test-run-nixbot-gitlab> gitlab # [ 3.062798] systemd[1]: Reached target Initrd Root Device.673vm-test-run-nixbot-gitlab> gitlab # [ 3.065852] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...674vm-test-run-nixbot-gitlab> gitlab # [ 3.067824] systemd[1]: Found device /dev/disk/by-label/nix-store.675vm-test-run-nixbot-gitlab> gitlab # [ 3.089845] systemd-fsck[241]: nixos: clean, [ 3.351731] cdrom: Uniform CD-ROM driver Revision: 3.20676vm-test-run-nixbot-gitlab> gitlab # 12/65536 files, 13019/262144 blocks677vm-test-run-nixbot-gitlab> gitlab # [ 3.100531] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.678vm-test-run-nixbot-gitlab> gitlab # [ 3.474853] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00679vm-test-run-nixbot-gitlab> gitlab # [ 3.480757] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10680vm-test-run-nixbot-gitlab> gitlab # [ 3.485177] usb 1-1: Product: QEMU USB Tablet681vm-test-run-nixbot-gitlab> gitlab # [ 3.487963] usb 1-1: Manufacturer: QEMU682vm-test-run-nixbot-gitlab> gitlab # [ 3.490434] usb 1-1: SerialNumber: 28754-0000:00:01.2-1683vm-test-run-nixbot-gitlab> gitlab # [ 3.551356] hid: raw HID events driver (C) Jiri Kosina684vm-test-run-nixbot-gitlab> gitlab # [ 3.560347] usbcore: registered new interface driver usbhid685vm-test-run-nixbot-gitlab> gitlab # [ 3.561779] usbhid: USB HID core driver686vm-test-run-nixbot-gitlab> gitlab # [ 3.568129] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2687vm-test-run-nixbot-gitlab> gitlab # [ 3.570867] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0688vm-test-run-nixbot-gitlab> gitlab # [ 3.447453] systemd[1]: Mounting /sysroot...689vm-test-run-nixbot-gitlab> gitlab # [ 3.900325] EXT4-fs (vda): mounted filesystem f474be07-6860-49a3-91a9-9c438fa888c6 r/w with ordered data mode. Quota mode: none.690vm-test-run-nixbot-gitlab> gitlab # [ 3.645429] systemd[1]: Mounted /sysroot.691vm-test-run-nixbot-gitlab> gitlab # [ 3.649731] systemd[1]: Reached target Initrd Root File System.692vm-test-run-nixbot-gitlab> gitlab # [ 3.659049] systemd[1]: Mounting /sysroot/nix/.ro-store...693vm-test-run-nixbot-gitlab> gitlab # [ 3.674148] systemd[1]: Mounting /sysroot/nix/.rw-store...694vm-test-run-nixbot-gitlab> gitlab # [ 3.682589] systemd[1]: Mounting /sysroot/run...695vm-test-run-nixbot-gitlab> gitlab # [ 3.950348] erofs (device vdb): mounted with root inode @ nid 128.696vm-test-run-nixbot-gitlab> gitlab # [ 3.691246] systemd[1]: Mounting /sysroot/tmp/shared...697vm-test-run-nixbot-gitlab> gitlab # [ 3.696671] systemd[1]: Mounting /sysroot/tmp/xchg...698vm-test-run-nixbot-gitlab> gitlab # [ 3.699705] systemd[1]: Starting Mountpoints Configured in the Real Root...699vm-test-run-nixbot-gitlab> gitlab # [ 3.708262] systemd[1]: Mounted /sysroot/nix/.ro-store.700vm-test-run-nixbot-gitlab> gitlab # [ 3.710092] systemd[1]: Mounted /sysroot/nix/.rw-store.701vm-test-run-nixbot-gitlab> gitlab # [ 3.711715] systemd[1]: Mounted /sysroot/run.702vm-test-run-nixbot-gitlab> gitlab # [ 3.713370] s[ 3.973936] 9p: Installing v9fs 9p2000 file system support703vm-test-run-nixbot-gitlab> gitlab # ystemd-sysroot-fstab-check[269]: /sysroot should be mounted in the initrd, will request daemon-reload.704vm-test-run-nixbot-gitlab> gitlab # [ 3.722085] systemd[1]: Mounted /sysroot/tmp/shared.705vm-test-run-nixbot-gitlab> gitlab # [ 3.723393] systemd[1]: Mounted /sysroot/tmp/xchg.706vm-test-run-nixbot-gitlab> gitlab # [ 3.724926] systemd[1]: Reload requested from client PID 269 ('systemd-sysroot') (unit initrd-parse-etc.service)...707vm-test-run-nixbot-gitlab> gitlab # [ 3.727570] systemd[1]: Reloading...708vm-test-run-nixbot-gitlab> gitlab # [ 4.007283] systemd[1]: Reloading finished in 281 ms.709vm-test-run-nixbot-gitlab> gitlab # [ 4.054559] systemd-sysroot-fstab-check[269]: Requesting initrd-fs.target/start/replace...710vm-test-run-nixbot-gitlab> gitlab # [ 4.204310] systemd-sysroot-fstab-check[269]: Requesting swap.target/start/replace...711vm-test-run-nixbot-gitlab> gitlab # [ 4.209240] systemd[1]: Starting rw-sysroot-nix-store.service...712vm-test-run-nixbot-gitlab> gitlab # [ 4.216307] systemd[1]: initrd-parse-etc.service: Deactivated successfully.713vm-test-run-nixbot-gitlab> gitlab # [ 4.226610] systemd[1]: Finished Mountpoints Configured in the Real Root.714vm-test-run-nixbot-gitlab> gitlab # [ 4.230150] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.715vm-test-run-nixbot-gitlab> gitlab # [ 4.237926] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.716vm-test-run-nixbot-gitlab> gitlab # [ 4.241737] systemd[1]: Finished rw-sysroot-nix-store.service.717vm-test-run-nixbot-gitlab> gitlab # [ 4.448733] systemd[1]: Mounting /sysroot/nix/store...718vm-test-run-nixbot-gitlab> gitlab # [ 4.505908] systemd[1]: Mounted /sysroot/nix/store.719vm-test-run-nixbot-gitlab> gitlab # [ 4.510824] systemd[1]: Reached target Initrd File Systems.720vm-test-run-nixbot-gitlab> gitlab # [ 4.519876] systemd[1]: Starting Find NixOS closure...721vm-test-run-nixbot-gitlab> gitlab # [ 4.528988] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...722vm-test-run-nixbot-gitlab> gitlab # [ 4.562231] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.723vm-test-run-nixbot-gitlab> gitlab # [ 4.567341] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.724vm-test-run-nixbot-gitlab> gitlab # [ 4.590634] systemd[1]: Finished Find NixOS closure.725vm-test-run-nixbot-gitlab> gitlab # [ 4.594338] systemd[1]: Reached target Initrd Default Target.726vm-test-run-nixbot-gitlab> gitlab # [ 4.598399] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...727vm-test-run-nixbot-gitlab> gitlab # [ 4.630501] systemd[1]: Stopped target Initrd Default Target.728vm-test-run-nixbot-gitlab> gitlab # [ 4.634392] systemd[1]: Stopped target Basic System.729vm-test-run-nixbot-gitlab> gitlab # [ 4.637618] systemd[1]: Stopped target Initrd Root Device.730vm-test-run-nixbot-gitlab> gitlab # [ 4.641122] systemd[1]: Stopped target Path Units.731vm-test-run-nixbot-gitlab> gitlab # [ 4.644242] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.732vm-test-run-nixbot-gitlab> gitlab # [ 4.648801] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.733vm-test-run-nixbot-gitlab> gitlab # [ 4.653531] systemd[1]: Stopped target Slice Units.734vm-test-run-nixbot-gitlab> gitlab # [ 4.656766] systemd[1]: Stopped target Socket Units.735vm-test-run-nixbot-gitlab> gitlab # [ 4.659935] systemd[1]: Stopped target System Initialization.736vm-test-run-nixbot-gitlab> gitlab # [ 4.663610] systemd[1]: Stopped target Swaps.737vm-test-run-nixbot-gitlab> gitlab # [ 4.666337] systemd[1]: Stopped target Timer Units.738vm-test-run-nixbot-gitlab> gitlab # [ 4.668845] systemd[1]: dbus.socket: Deactivated successfully.739vm-test-run-nixbot-gitlab> gitlab # [ 4.671653] systemd[1]: Closed D-Bus System Message Bus Socket.740vm-test-run-nixbot-gitlab> gitlab # [ 4.674535] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.741vm-test-run-nixbot-gitlab> gitlab # [ 4.678181] systemd[1]: Stopped Find NixOS closure.742vm-test-run-nixbot-gitlab> gitlab # [ 4.680743] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio743vm-test-run-nixbot-gitlab> gitlab # [ 4.685171] systemd[1]: Starting rw-sysroot-nix-store.service...744vm-test-run-nixbot-gitlab> gitlab # [ 4.686777] systemd[1]: systemd-sysctl.service: Deactivated successfully.745vm-test-run-nixbot-gitlab> gitlab # [ 4.688388] systemd[1]: Stopped Apply Kernel Variables.746vm-test-run-nixbot-gitlab> gitlab # [ 4.689810] systemd[1]: systemd-modules-load.service: Deactivated successfully.747vm-test-run-nixbot-gitlab> gitlab # [ 4.691607] systemd[1]: Stopped Load Kernel Modules.748vm-test-run-nixbot-gitlab> gitlab # [ 4.692922] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.749vm-test-run-nixbot-gitlab> gitlab # [ 4.694886] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.750vm-test-run-nixbot-gitlab> gitlab # [ 4.696815] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.751vm-test-run-nixbot-gitlab> gitlab # [ 4.698671] systemd[1]: Stopped Create System Files and Directories.752vm-test-run-nixbot-gitlab> gitlab # [ 4.700226] systemd[1]: Stopped target Local File Systems.753vm-test-run-nixbot-gitlab> gitlab # [ 4.701627] systemd[1]: Stopped target Preparation for Local File Systems.754vm-test-run-nixbot-gitlab> gitlab # [ 4.703526] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.755vm-test-run-nixbot-gitlab> gitlab # [ 4.705420] systemd[1]: Stopped Coldplug All udev Devices.756vm-test-run-nixbot-gitlab> gitlab # [ 4.708261] systemd[1]: Stopping Rule-based Manager for Device Events and Files...757vm-test-run-nixbot-gitlab> gitlab # [ 4.711065] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.758vm-test-run-nixbot-gitlab> gitlab # [ 4.713494] systemd[1]: Stopped Virtual Console Setup.759vm-test-run-nixbot-gitlab> gitlab # [ 4.717454] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.760vm-test-run-nixbot-gitlab> gitlab # [ 4.719913] systemd[1]: Finished rw-sysroot-nix-store.service.761vm-test-run-nixbot-gitlab> gitlab # [ 4.725304] systemd[1]: systemd-udevd.service: Deactivated successfully.762vm-test-run-nixbot-gitlab> gitlab # [ 4.727368] systemd[1]: Stopped Rule-based Manager for Device Events and Files.763vm-test-run-nixbot-gitlab> gitlab # [ 4.729401] systemd[1]: systemd-udevd.service: Consumed 1.106s CPU time over 2.188s wall clock time, 24.5M memory peak.764vm-test-run-nixbot-gitlab> gitlab # [ 4.735848] systemd[1]: initrd-cleanup.service: Deactivated successfully.765vm-test-run-nixbot-gitlab> gitlab # [ 4.739491] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.766vm-test-run-nixbot-gitlab> gitlab # [ 4.748168] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.767vm-test-run-nixbot-gitlab> gitlab # [ 4.750234] systemd[1]: Closed udev Control Socket.768vm-test-run-nixbot-gitlab> gitlab # [ 4.755486] systemd[1]: Starting Cleanup udev Database...769vm-test-run-nixbot-gitlab> gitlab # [ 4.757253] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.770vm-test-run-nixbot-gitlab> gitlab # [ 4.760203] systemd[1]: Stopped Create Static Device Nodes in /dev.771vm-test-run-nixbot-gitlab> gitlab # [ 4.764088] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.772vm-test-run-nixbot-gitlab> gitlab # [ 4.766236] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.773vm-test-run-nixbot-gitlab> gitlab # [ 4.769498] systemd[1]: kmod-static-nodes.service: Deactivated successfully.774vm-test-run-nixbot-gitlab> gitlab # [ 4.771214] systemd[1]: Stopped Create List of Static Device Nodes.775vm-test-run-nixbot-gitlab> gitlab # [ 4.776920] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.776vm-test-run-nixbot-gitlab> gitlab # [ 4.779056] systemd[1]: Finished Cleanup udev Database.777vm-test-run-nixbot-gitlab> gitlab # [ 4.782957] systemd[1]: Reached target Switch Root.778vm-test-run-nixbot-gitlab> gitlab # [ 4.786908] systemd[1]: Starting NixOS Activation...779vm-test-run-nixbot-gitlab> gitlab # [ 4.924285] initrd-nixos-activation-start[561]: booting system configuration /nix/store/3c8cfw70vhdlb9zswshrs4zpx9n4i5av-nixos-system-gitlab-test780vm-test-run-nixbot-gitlab> gitlab # [ 4.978734] initrd-nixos-activation-start[561]: running activation script...781vm-test-run-nixbot-gitlab> gitlab # [ 5.256705] initrd-nixos-activation-start[584]: setting up /etc...782vm-test-run-nixbot-gitlab> gitlab # [ 5.379240] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.783vm-test-run-nixbot-gitlab> gitlab # [ 5.384201] systemd[1]: Finished NixOS Activation.784vm-test-run-nixbot-gitlab> gitlab # [ 5.397858] systemd[1]: Starting Switch Root...785vm-test-run-nixbot-gitlab> gitlab # [ 5.420501] systemd[1]: Switching root.786vm-test-run-nixbot-gitlab> gitlab # [ 6.035971] systemd-journald[151]: Received SIGTERM from PID 1 (systemd).787vm-test-run-nixbot-gitlab> gitlab # [ 6.164429] NET: Registered PF_VSOCK protocol family788vm-test-run-nixbot-gitlab> gitlab # [ 6.535565] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)789vm-test-run-nixbot-gitlab> gitlab # [ 6.550395] systemd[1]: Detected virtualization kvm.790vm-test-run-nixbot-gitlab> gitlab # [ 6.553647] systemd[1]: Detected architecture x86-64.791vm-test-run-nixbot-gitlab> gitlab # [ 6.557043] systemd[1]: Detected first boot.792vm-test-run-nixbot-gitlab> gitlab # [ 6.561234] systemd[1]: Initializing machine ID from random generator.793vm-test-run-nixbot-gitlab> gitlab # [ 6.702937] systemd[1]: bpf-restrict-fs: LSM BPF program attached794vm-test-run-nixbot-gitlab> gitlab # [ 6.772137] systemd[1]: Applying preset policy.795vm-test-run-nixbot-gitlab> gitlab # [ 6.936559] systemd[1]: Populated /etc with preset unit settings.796vm-test-run-nixbot-gitlab> gitlab # [ 7.055549] systemd[1]: initrd-switch-root.service: Deactivated successfully.797vm-test-run-nixbot-gitlab> gitlab # [ 7.057929] systemd[1]: Stopped initrd-switch-root.service.798vm-test-run-nixbot-gitlab> gitlab # [ 7.061499] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.799vm-test-run-nixbot-gitlab> gitlab # [ 7.064797] systemd[1]: Created slice Slice /system/getty.800vm-test-run-nixbot-gitlab> gitlab # [ 7.066908] systemd[1]: Created slice GitLab DevOps Platform Slice.801vm-test-run-nixbot-gitlab> gitlab # [ 7.069262] systemd[1]: Created slice User and Session Slice.802vm-test-run-nixbot-gitlab> gitlab # [ 7.070863] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.803vm-test-run-nixbot-gitlab> gitlab # [ 7.072908] systemd[1]: Started Forward Password Requests to Wall Directory Watch.804vm-test-run-nixbot-gitlab> gitlab # [ 7.074785] systemd[1]: Expecting device /dev/hvc0...805vm-test-run-nixbot-gitlab> gitlab # [ 7.076060] systemd[1]: Expecting device /dev/ttyS0...806vm-test-run-nixbot-gitlab> gitlab # [ 7.077456] systemd[1]: Reached target Local Encrypted Volumes.807vm-test-run-nixbot-gitlab> gitlab # [ 7.078945] systemd[1]: Stopped target initrd-fs.target.808vm-test-run-nixbot-gitlab> gitlab # [ 7.080312] systemd[1]: Stopped target initrd-root-fs.target.809vm-test-run-nixbot-gitlab> gitlab # [ 7.081813] systemd[1]: Stopped target initrd-switch-root.target.810vm-test-run-nixbot-gitlab> gitlab # [ 7.083344] systemd[1]: Reached target Virtual Machines and Containers.811vm-test-run-nixbot-gitlab> gitlab # [ 7.085015] systemd[1]: Reached target Path Units.812vm-test-run-nixbot-gitlab> gitlab # [ 7.086318] systemd[1]: Reached target Remote File Systems.813vm-test-run-nixbot-gitlab> gitlab # [ 7.087779] systemd[1]: Reached target Slice Units.814vm-test-run-nixbot-gitlab> gitlab # [ 7.089153] systemd[1]: Reached target Swaps.815vm-test-run-nixbot-gitlab> gitlab # [ 7.091475] systemd[1]: Listening on Process Core Dump Socket.816vm-test-run-nixbot-gitlab> gitlab # [ 7.093621] systemd[1]: Listening on Credential Encryption/Decryption.817vm-test-run-nixbot-gitlab> gitlab # [ 7.097183] systemd[1]: Starting Journal Log Access Socket...818vm-test-run-nixbot-gitlab> gitlab # [ 7.099417] systemd[1]: Listening on Journal Audit Socket.819vm-test-run-nixbot-gitlab> gitlab # [ 7.101169] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.820vm-test-run-nixbot-gitlab> gitlab # [ 7.103025] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki821vm-test-run-nixbot-gitlab> gitlab # [ 7.105457] systemd[1]: Listening on udev Control Socket.822vm-test-run-nixbot-gitlab> gitlab # [ 7.110110] systemd[1]: Mounting Huge Pages File System...823vm-test-run-nixbot-gitlab> gitlab # [ 7.113078] systemd[1]: Mounting POSIX Message Queue File System...824vm-test-run-nixbot-gitlab> gitlab # [ 7.116149] systemd[1]: Mounting Kernel Debug File System...825vm-test-run-nixbot-gitlab> gitlab # [ 7.120601] systemd[1]: Mounting Kernel Trace File System...826vm-test-run-nixbot-gitlab> gitlab # [ 7.134390] systemd[1]: Starting Create List of Static Device Nodes...827vm-test-run-nixbot-gitlab> gitlab # [ 7.136174] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio828vm-test-run-nixbot-gitlab> gitlab # [ 7.146941] systemd[1]: Starting Load Kernel Module configfs...829vm-test-run-nixbot-gitlab> gitlab # [ 7.150928] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm830vm-test-run-nixbot-gitlab> gitlab # [ 7.156978] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore831vm-test-run-nixbot-gitlab> gitlab # [ 7.163915] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse832vm-test-run-nixbot-gitlab> gitlab # [ 7.178937] systemd[1]: Mounting FUSE Control File System...833vm-test-run-nixbot-gitlab> gitlab # [ 7.183047] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67834vm-test-run-nixbot-gitlab> gitlab # [ 7.200758] systemd[1]: Starting Journal Service...835vm-test-run-nixbot-gitlab> gitlab # [ 7.205955] systemd[1]: Starting Load Kernel Modules...836vm-test-run-nixbot-gitlab> gitlab # [ 7.225432] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...837vm-test-run-nixbot-gitlab> gitlab # [ 7.234107] systemd[1]: Starting Remount Root and Kernel File Systems...838vm-test-run-nixbot-gitlab> gitlab # [ 7.239244] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki839vm-test-run-nixbot-gitlab> gitlab # [ 7.245176] systemd-journald[781]: Collecting audit messages is enabled.840vm-test-run-nixbot-gitlab> gitlab # [ 7.250937] systemd[1]: Starting Coldplug All udev Devices...841vm-test-run-nixbot-gitlab> gitlab # [ 6.999149] systemd[1]: Queued start job for default target Multi-User System.842vm-test-run-nixbot-gitlab> gitlab # [ 7.004715] systemd[1]: Unnecessary job was removed for /dev/vdb.843vm-test-run-nixbot-gitlab> gitlab # [ 7.259927] systemd[1]: Started Journal Service.844vm-test-run-nixbot-gitlab> gitlab # [ 7.260803] EXT4-fs (vda): re-mounted f474be07-6860-49a3-91a9-9c438fa888c6.845vm-test-run-nixbot-gitlab> gitlab # [ 7.263856] loop: module loaded846vm-test-run-nixbot-gitlab> gitlab # [ 7.020848] systemd[1]: systemd-journald.service: Deactivated successfully.847vm-test-run-nixbot-gitlab> gitlab # [ 7.023214] systemd-modules-load[782]: Using 3 probe threads848vm-test-run-nixbot-gitlab> gitlab # [ 7.024980] systemd-modules-load[782]: Inserted module 'tls'849vm-test-run-nixbot-gitlab> gitlab # [ 7.026791] systemd-modules-load[782]: Inserted module 'loop'850vm-test-run-nixbot-gitlab> gitlab # [ 7.028512] systemd[1]: Listening on Journal Log Access Socket.851vm-test-run-nixbot-gitlab> gitlab # [ 7.030413] systemd[1]: Mounted Huge Pages File System.852vm-test-run-nixbot-gitlab> gitlab # [ 7.032219] systemd[1]: Mounted POSIX Message Queue File System.853vm-test-run-nixbot-gitlab> gitlab # [ 7.034160] systemd[1]: Mounted Kernel Debug File System.854vm-test-run-nixbot-gitlab> gitlab # [ 7.035703] systemd[1]: Mounted Kernel Trace File System.855vm-test-run-nixbot-gitlab> gitlab # [ 7.037423] systemd[1]: Finished Create List of Static Device Nodes.856vm-test-run-nixbot-gitlab> gitlab # [ 7.039115] systemd[1]: modprobe@configfs.service: Deactivated successfully.857vm-test-run-nixbot-gitlab> gitlab # [ 7.040863] systemd[1]: Finished Load Kernel Module configfs.858vm-test-run-nixbot-gitlab> gitlab # [ 7.042328] systemd[1]: Mounted FUSE Control File System.859vm-test-run-nixbot-gitlab> gitlab # [ 7.043680] systemd[1]: Finished Load Kernel Modules.860vm-test-run-nixbot-gitlab> gitlab # [ 7.044987] systemd[1]: Finished Remount Root and Kernel File Systems.861vm-test-run-nixbot-gitlab> gitlab # [ 7.046936] systemd-oomd[784]: No swap; memory pressure usage will be degraded862vm-test-run-nixbot-gitlab> gitlab # [ 7.049301] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.863vm-test-run-nixbot-gitlab> gitlab # [ 7.058419] systemd[1]: Mounting Kernel Configuration File System...864vm-test-run-nixbot-gitlab> gitlab # [ 7.062233] systemd[1]: Starting Firewall...865vm-test-run-nixbot-gitlab> gitlab # [ 7.063698] systemd[1]: Starting Flush Journal to Persistent Storage...866vm-test-run-nixbot-gitlab> gitlab # [ 7.065473] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore867vm-test-run-nixbot-gitlab> gitlab # [ 7.073851] systemd[1]: Starting Load/Save OS Random Seed...868vm-test-run-nixbot-gitlab> gitlab # [ 7.078164] systemd[1]: Starting Apply Kernel Variables...869vm-test-run-nixbot-gitlab> gitlab # [ 7.341365] systemd-journald[781]: Received client request to flush runtime journal.870vm-test-run-nixbot-gitlab> gitlab # [ 7.146498] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...871vm-test-run-nixbot-gitlab> gitlab # [ 7.149148] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki872vm-test-run-nixbot-gitlab> gitlab # [ 7.151547] systemd[1]: Mounted Kernel Configuration File System.873vm-test-run-nixbot-gitlab> gitlab # [ 7.153359] systemd[1]: Finished Load/Save OS Random Seed.874vm-test-run-nixbot-gitlab> gitlab # [ 7.154933] systemd[1]: Reached target First Boot Complete.875vm-test-run-nixbot-gitlab> gitlab # [ 7.156418] systemd[1]: Finished Apply Kernel Variables.876vm-test-run-nixbot-gitlab> gitlab # [ 7.157776] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.877vm-test-run-nixbot-gitlab> gitlab # [ 7.159599] systemd[1]: Starting Create Static Device Nodes in /dev...878vm-test-run-nixbot-gitlab> gitlab # [ 7.161191] systemd[1]: Finished Create Static Device Nodes in /dev.879vm-test-run-nixbot-gitlab> gitlab # [ 7.162826] systemd[1]: Reached target Preparation for Local File Systems.880vm-test-run-nixbot-gitlab> gitlab # [ 7.164827] systemd[1]: Starting Rule-based Manager for Device Events and Files...881vm-test-run-nixbot-gitlab> gitlab # [ 7.166803] systemd[1]: Finished Flush Journal to Persistent Storage.882vm-test-run-nixbot-gitlab> gitlab # [ 7.194343] systemd-udevd[828]: Using default interface naming scheme 'v260'.883vm-test-run-nixbot-gitlab> gitlab # [ 7.231690] systemd[1]: Finished Coldplug All udev Devices.884vm-test-run-nixbot-gitlab> gitlab # [ 7.235941] systemd[1]: Started Rule-based Manager for Device Events and Files.885vm-test-run-nixbot-gitlab> gitlab # [ 7.358188] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse886vm-test-run-nixbot-gitlab> gitlab # [ 7.361689] systemd[1]: Finished Firewall.887vm-test-run-nixbot-gitlab> gitlab # [ 7.388428] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.888vm-test-run-nixbot-gitlab> gitlab # [ 7.399418] (udev-worker)[923]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.889vm-test-run-nixbot-gitlab> gitlab # [ 7.403267] (udev-worker)[923]: Network interface NamePolicy= disabled on kernel command line.890vm-test-run-nixbot-gitlab> gitlab # [ 7.406171] (udev-worker)[917]: Network interface NamePolicy= disabled on kernel command line.891vm-test-run-nixbot-gitlab> gitlab # [ 7.414113] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.892vm-test-run-nixbot-gitlab> gitlab # [ 7.429584] systemd[1]: Condition check resulted in Virtio network device being skipped.893vm-test-run-nixbot-gitlab> gitlab # [ 7.431633] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore894vm-test-run-nixbot-gitlab> gitlab # [ 7.434257] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67895vm-test-run-nixbot-gitlab> gitlab # [ 7.438294] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore896vm-test-run-nixbot-gitlab> gitlab # [ 7.441226] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki897vm-test-run-nixbot-gitlab> gitlab # [ 7.443634] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki898vm-test-run-nixbot-gitlab> gitlab # [ 7.731429] mousedev: PS/2 mouse device common for all mice899vm-test-run-nixbot-gitlab> gitlab # [ 7.746635] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input3900vm-test-run-nixbot-gitlab> gitlab # [ 7.754082] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input4901vm-test-run-nixbot-gitlab> gitlab # [ 7.768407] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console902vm-test-run-nixbot-gitlab> gitlab # [ 7.768800] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0903vm-test-run-nixbot-gitlab> gitlab # [ 7.772732] ACPI: button: Power Button [PWRF]904vm-test-run-nixbot-gitlab> gitlab # [ 7.781479] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD905vm-test-run-nixbot-gitlab> gitlab # [ 7.792922] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6906vm-test-run-nixbot-gitlab> gitlab # [ 7.798848] Console: switching to colour dummy device 80x25907vm-test-run-nixbot-gitlab> gitlab # [ 7.799860] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5908vm-test-run-nixbot-gitlab> gitlab # [ 7.801512] [drm] Found bochs VGA, ID 0xb0c5.909vm-test-run-nixbot-gitlab> gitlab # [ 7.803247] rtc_cmos 00:05: RTC can wake from S4910vm-test-run-nixbot-gitlab> gitlab # [ 7.804038] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.911vm-test-run-nixbot-gitlab> gitlab # [ 7.810192] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic912vm-test-run-nixbot-gitlab> gitlab # [ 7.810291] parport_pc 00:03: reported by Plug and Play ACPI913vm-test-run-nixbot-gitlab> gitlab # [ 7.812079] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0914vm-test-run-nixbot-gitlab> gitlab # [ 7.814984] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]915vm-test-run-nixbot-gitlab> gitlab # [ 7.818529] fbcon: bochs-drmdrmfb (fb0) is primary device916vm-test-run-nixbot-gitlab> gitlab # [ 7.819039] Floppy drive(s): fd0 is 2.88M AMI BIOS917vm-test-run-nixbot-gitlab> gitlab # [ 7.827843] rtc_cmos 00:05: registered as rtc0918vm-test-run-nixbot-gitlab> gitlab # [ 7.828025] rtc_cmos 00:05: setting system clock to 2026-06-30T07:09:44 UTC (1782803384)919vm-test-run-nixbot-gitlab> gitlab # [ 7.828175] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs920vm-test-run-nixbot-gitlab> gitlab # [ 7.828985] Console: switching to colour frame buffer device 160x50921vm-test-run-nixbot-gitlab> gitlab # [ 7.832986] FDC 0 is a S82078B922vm-test-run-nixbot-gitlab> gitlab # [ 7.834032] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device923vm-test-run-nixbot-gitlab> gitlab # [ 7.669189] systemd[1]: Starting Virtual Console Setup...924vm-test-run-nixbot-gitlab> gitlab # [ 7.677473] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.925vm-test-run-nixbot-gitlab> gitlab # [ 7.683711] systemd[1]: Stopped Virtual Console Setup.926vm-test-run-nixbot-gitlab> gitlab # [ 7.695320] systemd[1]: Starting Virtual Console Setup...927vm-test-run-nixbot-gitlab> gitlab # [ 7.716291] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.928vm-test-run-nixbot-gitlab> gitlab # [ 7.720283] systemd[1]: Stopped Virtual Console Setup.929vm-test-run-nixbot-gitlab> gitlab # [ 7.732075] systemd[1]: Starting Virtual Console Setup...930vm-test-run-nixbot-gitlab> gitlab # [ 7.996793] ppdev: user-space parallel port driver931vm-test-run-nixbot-gitlab> gitlab # [ 7.997812] kvm_amd: TSC scaling supported932vm-test-run-nixbot-gitlab> gitlab # [ 7.998986] kvm_amd: Nested Virtualization enabled933vm-test-run-nixbot-gitlab> gitlab # [ 8.000013] kvm_amd: Nested Paging enabled934vm-test-run-nixbot-gitlab> gitlab # [ 8.000874] kvm_amd: LBR virtualization supported935vm-test-run-nixbot-gitlab> gitlab # [ 8.001842] kvm_amd: Virtual VMLOAD VMSAVE supported936vm-test-run-nixbot-gitlab> gitlab # [ 8.002865] kvm_amd: Virtual GIF supported937vm-test-run-nixbot-gitlab> gitlab # [ 8.003634] kvm_amd: Virtual NMI enabled938vm-test-run-nixbot-gitlab> gitlab # [ 8.055299] EDAC MC: Ver: 3.0.0939vm-test-run-nixbot-gitlab> gitlab # [ 7.797046] systemd[1]: Mounting /run/wrappers...940vm-test-run-nixbot-gitlab> gitlab # [ 7.818858] systemd-vconsole-setup[954]: Configuration of first virtual console was skipped, ignoring remaining ones.941vm-test-run-nixbot-gitlab> gitlab # [ 7.823445] systemd[1]: Finished Virtual Console Setup.942vm-test-run-nixbot-gitlab> gitlab # [ 7.825044] systemd[1]: Mounted /run/wrappers.943vm-test-run-nixbot-gitlab> gitlab # [ 7.826705] systemd[1]: Reached target Local File Systems.944vm-test-run-nixbot-gitlab> gitlab # [ 7.828325] systemd[1]: Listening on Boot Loader Control Service Socket.945vm-test-run-nixbot-gitlab> gitlab # [ 7.830241] systemd[1]: Starting register-nix-paths.service...946vm-test-run-nixbot-gitlab> gitlab # [ 7.831932] systemd[1]: Starting Create SUID/SGID Wrappers...947vm-test-run-nixbot-gitlab> gitlab # [ 7.833502] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.948vm-test-run-nixbot-gitlab> gitlab # [ 7.835787] systemd[1]: Starting Save Transient machine-id to Disk...949vm-test-run-nixbot-gitlab> gitlab # [ 7.850564] systemd[1]: Starting Create System Files and Directories...950vm-test-run-nixbot-gitlab> gitlab # [ 7.876267] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.951vm-test-run-nixbot-gitlab> gitlab # [ 7.878921] systemd[1]: Finished Save Transient machine-id to Disk.952vm-test-run-nixbot-gitlab> gitlab # [ 7.885160] systemd[1]: Finished Create System Files and Directories.953vm-test-run-nixbot-gitlab> gitlab # [ 7.888656] systemd[1]: Starting Rebuild Journal Catalog...954vm-test-run-nixbot-gitlab> gitlab # [ 7.892298] systemd[1]: Starting Record System Boot/Shutdown in UTMP...955vm-test-run-nixbot-gitlab> gitlab # [ 7.910588] systemd[1]: Finished Record System Boot/Shutdown in UTMP.956vm-test-run-nixbot-gitlab> gitlab # [ 7.925946] systemd[1]: Finished Rebuild Journal Catalog.957vm-test-run-nixbot-gitlab> gitlab # [ 7.927956] systemd[1]: Starting Update is Completed...958vm-test-run-nixbot-gitlab> gitlab # [ 7.945756] systemd[1]: Finished Update is Completed.959vm-test-run-nixbot-gitlab> gitlab # [ 7.993643] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.960vm-test-run-nixbot-gitlab> gitlab # [ 7.995773] systemd[1]: Finished Create SUID/SGID Wrappers.961vm-test-run-nixbot-gitlab> gitlab # [ 7.997348] systemd[1]: suid-sgid-wrappers.service: Consumed 120ms CPU time over 163ms wall clock time, 4.7M memory peak, 1.1M read from disk.962vm-test-run-nixbot-gitlab> gitlab # [ 8.245904] systemd[1]: Finished register-nix-paths.service.963vm-test-run-nixbot-gitlab> gitlab # [ 8.250577] systemd[1]: Reached target System Initialization.964vm-test-run-nixbot-gitlab> gitlab # [ 8.254919] systemd[1]: Started Discard unused filesystem blocks once a week.965vm-test-run-nixbot-gitlab> gitlab # [ 8.259721] systemd[1]: Started logrotate.timer.966vm-test-run-nixbot-gitlab> gitlab # [ 8.262745] systemd[1]: Started Daily Cleanup of Temporary Directories.967vm-test-run-nixbot-gitlab> gitlab # [ 8.266876] systemd[1]: Reached target Timer Units.968vm-test-run-nixbot-gitlab> gitlab # [ 8.270144] systemd[1]: Listening on D-Bus System Message Bus Socket.969vm-test-run-nixbot-gitlab> gitlab # [ 8.274266] systemd[1]: Listening on Nix Daemon Socket.970vm-test-run-nixbot-gitlab> gitlab # [ 8.288988] systemd[1]: Starting nixbot CI service socket...971vm-test-run-nixbot-gitlab> gitlab # [ 8.292666] systemd[1]: Listening on Hostname Service Socket.972vm-test-run-nixbot-gitlab> gitlab # [ 8.296368] systemd[1]: Starting D-Bus System Message Bus...973vm-test-run-nixbot-gitlab> gitlab # [ 8.299967] systemd[1]: Listening on nixbot CI service socket.974vm-test-run-nixbot-gitlab> gitlab # [ 8.303727] systemd[1]: Reached target Socket Units.975vm-test-run-nixbot-gitlab> gitlab # [ 8.336977] dbus-broker-launch[1044]: Looking up NSS user entry for 'systemd-timesync'...976vm-test-run-nixbot-gitlab> gitlab # [ 8.342895] dbus-broker-launch[1044]: NSS returned no entry for 'systemd-timesync'977vm-test-run-nixbot-gitlab> gitlab # [ 8.347487] dbus-broker-launch[1044]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"978vm-test-run-nixbot-gitlab> gitlab # [ 8.356845] systemd[1]: Started D-Bus System Message Bus.979vm-test-run-nixbot-gitlab> gitlab # [ 8.372375] dbus-broker-launch[1044]: Ready980vm-test-run-nixbot-gitlab> gitlab # [ 8.376583] systemd[1]: Reached target Basic System.981vm-test-run-nixbot-gitlab> gitlab # [ 8.380789] systemd[1]: Started backdoor.service.982vm-test-run-nixbot-gitlab> gitlab # [ 8.384444] systemd[1]: Starting gitlab-config.service...983vm-test-run-nixbot-gitlab> gitlab # [ 8.389176] systemd[1]: Starting Import lastlog data into lastlog2 database...984vm-test-run-nixbot-gitlab> gitlab # [ 8.393133] systemd[1]: Starting Logrotate configuration check...985vm-test-run-nixbot-gitlab> gitlab # [ 8.402073] systemd[1]: Starting Name Service Cache Daemon (nsncd)...986vm-test-run-nixbot-gitlab> gitlab # [ 8.407354] systemd[1]: Starting Post-Boot Actions...987vm-test-run-nixbot-gitlab> gitlab # [ 8.411059] systemd[1]: Started Reset console on configuration changes.988vm-test-run-nixbot-gitlab> gitlab # [ 8.414275] systemd[1]: Starting resolvconf update...989vm-test-run-nixbot-gitlab> gitlab # connecting to host...990vm-test-run-nixbot-gitlab> gitlab # [ 8.423782] logrotate[1049]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.991vm-test-run-nixbot-gitlab> gitlab # [ 8.426421] logrotate[1049]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf992vm-test-run-nixbot-gitlab> gitlab # [ 8.430170] logrotate[1049]: note: 'monthly' overrides previously specified 'weekly'993vm-test-run-nixbot-gitlab> gitlab # [ 8.431664] logrotate[1049]: Reading state from file: /var/lib/logrotate.status994vm-test-run-nixbot-gitlab> gitlab # [ 8.432903] logrotate[1049]: state file /var/lib/logrotate.status does not exist995vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'996vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell997vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.26 seconds)998vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.26 seconds)999vm-test-run-nixbot-gitlab> gitlab # [ 8.434125] logrotate[1049]: Allocating hash table for state file, size 64 entries1000vm-test-run-nixbot-gitlab> gitlab # [ 8.435102] logrotate[1049]: Handling 4 logs1001vm-test-run-nixbot-gitlab> gitlab # [ 8.435960] logrotate[1049]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1002vm-test-run-nixbot-gitlab> gitlab # [ 8.437086] logrotate[1049]: considering log /var/log/btmp1003vm-test-run-nixbot-gitlab> gitlab # [ 8.437892] logrotate[1049]: Creating new state1004vm-test-run-nixbot-gitlab> gitlab # [ 8.439048] logrotate[1049]: Now: 2026-06-30 07:091005vm-test-run-nixbot-gitlab> gitlab # [ 8.439869] logrotate[1049]: Last rotated at 2026-06-30 07:001006vm-test-run-nixbot-gitlab> gitlab # [ 8.440953] logrotate[1049]: log does not need rotating (log has already been rotated)1007vm-test-run-nixbot-gitlab> gitlab # [ 8.441788] logrotate[1049]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1008vm-test-run-nixbot-gitlab> gitlab # [ 8.442965] logrotate[1049]: considering log /var/log/wtmp1009vm-test-run-nixbot-gitlab> gitlab # [ 8.444054] logrotate[1049]: Creating new state1010vm-test-run-nixbot-gitlab> gitlab # [ 8.444794] logrotate[1049]: Now: 2026-06-30 07:091011vm-test-run-nixbot-gitlab> gitlab # [ 8.445904] logrotate[1049]: Last rotated at 2026-06-30 07:001012vm-test-run-nixbot-gitlab> gitlab # [ 8.447131] logrotate[1049]: log does not need rotating (log has already been rotated)1013vm-test-run-nixbot-gitlab> gitlab # [ 8.447999] logrotate[1049]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed1014vm-test-run-nixbot-gitlab> gitlab # [ 8.448872] logrotate[1049]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1049)1015vm-test-run-nixbot-gitlab> gitlab # [ 8.449990] logrotate[1049]: considering log /var/gitlab/state/log/*.log1016vm-test-run-nixbot-gitlab> gitlab # [ 8.450905] logrotate[1049]: log /var/gitlab/state/log/*.log does not exist -- skipping1017vm-test-run-nixbot-gitlab> gitlab # [ 8.451493] logrotate[1049]: Creating new state1018vm-test-run-nixbot-gitlab> gitlab # [ 8.452351] logrotate[1049]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1049)1019vm-test-run-nixbot-gitlab> gitlab # [ 8.453246] logrotate[1049]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed1020vm-test-run-nixbot-gitlab> gitlab # [ 8.454141] logrotate[1049]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1049)1021vm-test-run-nixbot-gitlab> gitlab # [ 8.454976] logrotate[1049]: considering log /var/log/nginx/*.log1022vm-test-run-nixbot-gitlab> gitlab # [ 8.456169] logrotate[1049]: log /var/log/nginx/*.log does not exist -- skipping1023vm-test-run-nixbot-gitlab> gitlab # [ 8.457066] logrotate[1049]: Creating new state1024vm-test-run-nixbot-gitlab> gitlab # [ 8.457642] logrotate[1049]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1049)1025vm-test-run-nixbot-gitlab> gitlab # [ 8.481170] systemd[1]: Finished Post-Boot Actions.1026vm-test-run-nixbot-gitlab> gitlab # [ 8.485974] systemd[1]: Finished Logrotate configuration check.1027vm-test-run-nixbot-gitlab> gitlab # [ 8.489258] systemd[1]: Finished Import lastlog data into lastlog2 database.1028vm-test-run-nixbot-gitlab> gitlab # [ 8.513270] systemd[1]: Started Name Service Cache Daemon (nsncd).1029vm-test-run-nixbot-gitlab> gitlab # [ 8.514757] nsncd[1050]: Jun 30 07:09:45.444 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1030vm-test-run-nixbot-gitlab> gitlab # [ 8.515439] systemd[1]: Reached target Host and Network Name Lookups.1031vm-test-run-nixbot-gitlab> gitlab # [ 8.516615] systemd[1]: Reached target User and Group Name Lookups.1032vm-test-run-nixbot-gitlab> gitlab # [ 8.518452] systemd[1]: Starting User Login Management...1033vm-test-run-nixbot-gitlab> gitlab # [ 8.547171] systemd[1]: Stopped target Host and Network Name Lookups.1034vm-test-run-nixbot-gitlab> gitlab # [ 8.549257] systemd[1]: Stopping Host and Network Name Lookups...1035vm-test-run-nixbot-gitlab> gitlab # [ 8.551115] systemd[1]: Stopped target User and Group Name Lookups.1036vm-test-run-nixbot-gitlab> gitlab # [ 8.552892] systemd[1]: Stopping User and Group Name Lookups...1037vm-test-run-nixbot-gitlab> gitlab # [ 8.554604] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...1038vm-test-run-nixbot-gitlab> gitlab # [ 8.556375] systemd[1]: nscd.service: Deactivated successfully.1039vm-test-run-nixbot-gitlab> gitlab # [ 8.558223] systemd[1]: Stopped Name Service Cache Daemon (nsncd).1040vm-test-run-nixbot-gitlab> gitlab # [ 8.559971] systemd[1]: nscd.service: Consumed 24ms CPU time over 153ms wall clock time, 7.8M memory peak, 5.6M read from disk.1041vm-test-run-nixbot-gitlab> gitlab # [ 8.563109] systemd[1]: Starting Name Service Cache Daemon (nsncd)...1042vm-test-run-nixbot-gitlab> gitlab # [ 8.565497] systemd-logind[1103]: New seat seat0.1043vm-test-run-nixbot-gitlab> gitlab # [ 8.569605] systemd-logind[1103]: Watching system buttons on /dev/input/event3 (Power Button)1044vm-test-run-nixbot-gitlab> gitlab # [ 8.572325] systemd-logind[1103]: Watching system buttons on /dev/input/event2 (QEMU Virtio Keyboard)1045vm-test-run-nixbot-gitlab> gitlab # [ 8.574659] systemd-logind[1103]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)1046vm-test-run-nixbot-gitlab> gitlab # [ 8.577049] systemd[1]: Started User Login Management.1047vm-test-run-nixbot-gitlab> gitlab # [ 8.582610] systemd[1]: Starting linger-users.service...1048vm-test-run-nixbot-gitlab> gitlab # [ 8.587165] systemd[1]: Finished resolvconf update.1049vm-test-run-nixbot-gitlab> gitlab # [ 8.589566] systemd[1]: Reached target Preparation for Network.1050vm-test-run-nixbot-gitlab> gitlab # [ 8.596416] nsncd[1136]: Jun 30 07:09:45.527 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1051vm-test-run-nixbot-gitlab> gitlab # [ 8.600294] systemd[1]: Starting DHCP Client...1052vm-test-run-nixbot-gitlab> gitlab # [ 8.603373] systemd[1]: Starting Address configuration of eth1...1053vm-test-run-nixbot-gitlab> gitlab # [ 8.606506] systemd[1]: Starting Extra networking commands....1054vm-test-run-nixbot-gitlab> gitlab # [ 8.609787] systemd[1]: Started Name Service Cache Daemon (nsncd).1055vm-test-run-nixbot-gitlab> gitlab # [ 8.610916] systemd[1]: linger-users.service: Deactivated successfully.1056vm-test-run-nixbot-gitlab> gitlab # [ 8.616190] systemd[1]: Finished linger-users.service.1057vm-test-run-nixbot-gitlab> gitlab # [ 8.619974] systemd[1]: Reached target Host and Network Name Lookups.1058vm-test-run-nixbot-gitlab> gitlab # [ 8.620927] systemd[1]: Reached target User and Group Name Lookups.1059vm-test-run-nixbot-gitlab> gitlab # [ 8.636335] network-addresses-eth1-start[1161]: adding address 192.168.1.1/24... done1060vm-test-run-nixbot-gitlab> gitlab # [ 8.641114] network-addresses-eth1-start[1161]: adding address 2001:db8:1::1/64... done1061vm-test-run-nixbot-gitlab> gitlab # [ 8.651146] systemd[1]: Finished Address configuration of eth1.1062vm-test-run-nixbot-gitlab> gitlab # [ 8.682581] systemd[1]: Finished Extra networking commands..1063vm-test-run-nixbot-gitlab> gitlab # [ 8.684509] systemd[1]: Reached target Network.1064vm-test-run-nixbot-gitlab> gitlab # [ 8.689132] systemd[1]: Starting gitlab-workhorse.service...1065vm-test-run-nixbot-gitlab> gitlab # [ 8.692457] systemd[1]: Starting Nginx Web Server...1066vm-test-run-nixbot-gitlab> gitlab # [ 8.697383] systemd[1]: Starting PostgreSQL Server...1067vm-test-run-nixbot-gitlab> gitlab # [ 8.701527] systemd[1]: Starting Redis Server - redis-gitlab...1068vm-test-run-nixbot-gitlab> gitlab # [ 8.706151] systemd[1]: Starting Permit User Sessions...1069vm-test-run-nixbot-gitlab> gitlab # [ 8.707638] dhcpcd[1187]: dhcpcd-10.3.2 starting1070vm-test-run-nixbot-gitlab> gitlab # [ 8.711393] dhcpcd[1225]: dev: loaded udev1071vm-test-run-nixbot-gitlab> gitlab # [ 8.980202] 8021q: 802.1Q VLAN Support v1.81072vm-test-run-nixbot-gitlab> gitlab # [ 8.983908] 8021q: adding VLAN 0 to HW filter on device eth11073vm-test-run-nixbot-gitlab> gitlab # [ 8.733387] systemd[1]: Finished Permit User Sessions.1074vm-test-run-nixbot-gitlab> gitlab # [ 8.739527] systemd[1]: Started Getty on tty1.1075vm-test-run-nixbot-gitlab> gitlab # [ 8.742419] systemd[1]: Reached target Login Prompts.1076vm-test-run-nixbot-gitlab> gitlab # [ 8.748805] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1077vm-test-run-nixbot-gitlab> gitlab # [ 9.091620] cfg80211: Loading compiled-in X.509 certificates for regulatory database1078vm-test-run-nixbot-gitlab> gitlab # [ 9.096727] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'1079vm-test-run-nixbot-gitlab> gitlab # [ 9.098221] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'1080vm-test-run-nixbot-gitlab> gitlab # [ 9.100598] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -21081vm-test-run-nixbot-gitlab> gitlab # [ 9.103046] cfg80211: failed to load regulatory.db1082vm-test-run-nixbot-gitlab> gitlab # [ 8.844189] nginx-pre-start[1243]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok1083vm-test-run-nixbot-gitlab> gitlab # [ 8.844886] nginx-pre-start[1243]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful1084vm-test-run-nixbot-gitlab> gitlab # [ 8.856559] systemd[1]: Started Nginx Web Server.1085vm-test-run-nixbot-gitlab> gitlab # [ 8.863249] postgresql-pre-start[1247]: The files belonging to this database system will be owned by user "postgres".1086vm-test-run-nixbot-gitlab> gitlab # [ 8.864195] postgresql-pre-start[1247]: This user must also own the server process.1087vm-test-run-nixbot-gitlab> gitlab # [ 8.869073] postgresql-pre-start[1247]: The database cluster will be initialized with locale "en_US.UTF-8".1088vm-test-run-nixbot-gitlab> gitlab # [ 8.869890] postgresql-pre-start[1247]: The default database encoding has accordingly been set to "UTF8".1089vm-test-run-nixbot-gitlab> gitlab # [ 8.872674] postgresql-pre-start[1247]: The default text search configuration will be set to "english".1090vm-test-run-nixbot-gitlab> gitlab # [ 8.873681] postgresql-pre-start[1247]: Data page checksums are disabled.1091vm-test-run-nixbot-gitlab> gitlab # [ 8.875763] postgresql-pre-start[1247]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1092vm-test-run-nixbot-gitlab> gitlab # [ 8.877209] postgresql-pre-start[1247]: creating subdirectories ... ok1093vm-test-run-nixbot-gitlab> gitlab # [ 8.879417] postgresql-pre-start[1247]: selecting dynamic shared memory implementation ... posix1094vm-test-run-nixbot-gitlab> gitlab # [ 8.882646] redis-server[1245]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1095vm-test-run-nixbot-gitlab> gitlab # [ 8.885065] redis-server[1245]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1096vm-test-run-nixbot-gitlab> gitlab # [ 8.887874] systemd[1]: Started Redis Server - redis-gitlab.1097vm-test-run-nixbot-gitlab> gitlab # [ 8.889123] redis-server[1245]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1245, just started1098vm-test-run-nixbot-gitlab> gitlab # [ 8.890976] redis-server[1245]: Configuration loaded1099vm-test-run-nixbot-gitlab> gitlab # [ 8.894105] redis[1245]: monotonic clock: POSIX clock_gettime1100vm-test-run-nixbot-gitlab> gitlab # [ 8.894931] redis[1245]: Running mod[ 9.158415] 8021q: adding VLAN 0 to HW filter on device eth01101vm-test-run-nixbot-gitlab> gitlab # e=standalone, port=0.1102vm-test-run-nixbot-gitlab> gitlab # [ 8.898085] redis[1245]: Server initialized1103vm-test-run-nixbot-gitlab> gitlab # [ 8.899966] redis[1245]: Ready to accept connections unix1104vm-test-run-nixbot-gitlab> gitlab # [ 8.902661] redis[1245]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1105vm-test-run-nixbot-gitlab> gitlab # [ 8.903865] dhcpcd[1225]: eth0: waiting for carrier1106vm-test-run-nixbot-gitlab> gitlab # [ 8.905488] dhcpcd[1225]: eth0: carrier acquired1107vm-test-run-nixbot-gitlab> gitlab # [ 8.913863] dhcpcd[1225]: DUID 00:01:00:01:31:d6:28:39:52:54:00:12:34:561108vm-test-run-nixbot-gitlab> gitlab # [ 8.914712] dhcpcd[1225]: eth0: IAID 00:12:34:561109vm-test-run-nixbot-gitlab> gitlab # [ 8.915994] dhcpcd[1225]: eth0: adding address fe80::5054:ff:fe12:34561110vm-test-run-nixbot-gitlab> gitlab # [ 8.918166] postgresql-pre-start[1247]: selecting default "max_connections" ... 1001111vm-test-run-nixbot-gitlab> gitlab # [ 8.959423] postgresql-pre-start[1247]: selecting default "shared_buffers" ... 128MB1112vm-test-run-nixbot-gitlab> gitlab # [ 9.178149] systemd[1]: Started gitlab-workhorse.service.1113vm-test-run-nixbot-gitlab> gitlab # [ 9.239418] dhcpcd[1225]: eth0: soliciting a DHCP lease1114vm-test-run-nixbot-gitlab> gitlab # [ 9.506125] NET: Registered PF_PACKET protocol family1115vm-test-run-nixbot-gitlab> gitlab # [ 9.252797] dhcpcd[1225]: eth0: offered 10.0.2.15 from 10.0.2.21116vm-test-run-nixbot-gitlab> gitlab # [ 9.260317] dhcpcd[1225]: eth0: probing address 10.0.2.15/241117vm-test-run-nixbot-gitlab> gitlab # [ 9.339752] postgresql-pre-start[1247]: selecting default time zone ... UTC1118vm-test-run-nixbot-gitlab> gitlab # [ 9.341707] postgresql-pre-start[1247]: creating configuration files ... ok1119vm-test-run-nixbot-gitlab> gitlab # [ 9.466176] gitlab-workhorse[1280]: time="2026-06-30T07:09:46Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41120vm-test-run-nixbot-gitlab> gitlab # [ 9.466826] gitlab-workhorse[1280]: time="2026-06-30T07:09:46Z" level=info msg="Using redis/go-redis"1121vm-test-run-nixbot-gitlab> gitlab # [ 9.468049] gitlab-workhorse[1280]: time="2026-06-30T07:09:46Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1122vm-test-run-nixbot-gitlab> gitlab # [ 9.538171] postgresql-pre-start[1247]: running bootstrap script ... ok1123vm-test-run-nixbot-gitlab> gitlab # [ 10.005951] postgresql-pre-start[1247]: performing post-bootstrap initialization ... ok1124vm-test-run-nixbot-gitlab> gitlab # [ 10.458286] postgresql-pre-start[1247]: syncing data to disk ... ok1125vm-test-run-nixbot-gitlab> gitlab # [ 10.462458] postgresql-pre-start[1247]: initdb: warning: enabling "trust" authentication for local connections1126vm-test-run-nixbot-gitlab> gitlab # [ 10.468330] postgresql-pre-start[1247]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1127vm-test-run-nixbot-gitlab> gitlab # [ 10.477343] postgresql-pre-start[1247]: Success. You can now start the database server using:1128vm-test-run-nixbot-gitlab> gitlab # [ 10.482296] postgresql-pre-start[1247]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1129vm-test-run-nixbot-gitlab> gitlab # [ 10.588717] postgres[1306]: [1306] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1130vm-test-run-nixbot-gitlab> gitlab # [ 10.596427] postgres[1306]: [1306] LOG: listening on IPv6 address "::1", port 54321131vm-test-run-nixbot-gitlab> gitlab # [ 10.601874] postgres[1306]: [1306] LOG: listening on IPv4 address "127.0.0.1", port 54321132vm-test-run-nixbot-gitlab> gitlab # [ 10.606538] postgres[1306]: [1306] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1133vm-test-run-nixbot-gitlab> gitlab # [ 10.611407] postgres[1317]: [1317] LOG: database system was shut down at 2026-06-30 07:09:46 GMT1134vm-test-run-nixbot-gitlab> gitlab # [ 10.615685] postgres[1306]: [1306] LOG: database system is ready to accept connections1135vm-test-run-nixbot-gitlab> gitlab # [ 10.620342] systemd[1]: Started PostgreSQL Server.1136vm-test-run-nixbot-gitlab> gitlab # [ 10.624486] systemd[1]: Starting PostgreSQL Setup Scripts...1137vm-test-run-nixbot-gitlab> gitlab # [ 10.764596] postgresql-setup-start[1333]: CREATE DATABASE1138vm-test-run-nixbot-gitlab> gitlab # [ 10.802582] postgresql-setup-start[1343]: CREATE ROLE1139vm-test-run-nixbot-gitlab> gitlab # [ 10.819841] postgresql-setup-start[1345]: ALTER DATABASE1140vm-test-run-nixbot-gitlab> gitlab # [ 10.845935] postgresql-setup-start[1350]: CREATE ROLE1141vm-test-run-nixbot-gitlab> gitlab # [ 10.850334] systemd[1]: Finished PostgreSQL Setup Scripts.1142vm-test-run-nixbot-gitlab> gitlab # [ 10.852796] systemd[1]: Reached target PostgreSQL.1143vm-test-run-nixbot-gitlab> gitlab # [ 10.856266] systemd[1]: Starting gitlab-postgresql.service...1144vm-test-run-nixbot-gitlab> gitlab # [ 10.863842] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1338]: mkdir -p /var/gitlab/state/home/.ssh: OK1145vm-test-run-nixbot-gitlab> gitlab # [ 10.868382] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1338]: chmod 700 /var/gitlab/state/home/.ssh: OK1146vm-test-run-nixbot-gitlab> gitlab # [ 10.917605] systemd[1]: Finished gitlab-config.service.1147vm-test-run-nixbot-gitlab> gitlab # [ 10.921324] systemd[1]: Started gitaly.service.1148vm-test-run-nixbot-gitlab> gitlab # [ 10.922440] systemd[1]: Starting gitlab-db-config.service...1149vm-test-run-nixbot-gitlab> gitlab # [ 10.967596] gitlab-postgresql-start[1366]: CREATE DATABASE1150vm-test-run-nixbot-gitlab> gitlab # [ 11.000872] gitlab-postgresql-start[1379]: CREATE EXTENSION1151vm-test-run-nixbot-gitlab> gitlab # [ 11.035294] gitlab-postgresql-start[1381]: CREATE EXTENSION1152vm-test-run-nixbot-gitlab> gitlab # [ 11.038244] systemd[1]: Finished gitlab-postgresql.service.1153vm-test-run-nixbot-gitlab> gitlab # [ 11.045791] dhcpcd[1225]: eth0: soliciting an IPv6 router1154vm-test-run-nixbot-gitlab> gitlab # [ 11.047244] dhcpcd[1225]: eth0: Router Advertisement from fe80::21155vm-test-run-nixbot-gitlab> gitlab # [ 11.048184] dhcpcd[1225]: eth0: adding address fec0::5054:ff:fe12:3456/641156vm-test-run-nixbot-gitlab> gitlab # [ 11.048823] dhcpcd[1225]: eth0: adding route to fec0::/641157vm-test-run-nixbot-gitlab> gitlab # [ 11.049645] dhcpcd[1225]: eth0: adding default route via fe80::21158vm-test-run-nixbot-gitlab> gitlab # [ 11.551892] gitaly[1372]: time="2026-06-30T07:09:48.483Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13721159vm-test-run-nixbot-gitlab> gitlab # [ 11.553489] gitaly[1372]: time="2026-06-30T07:09:48.483Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13721160vm-test-run-nixbot-gitlab> gitlab # [ 11.555417] gitaly[1372]: time="2026-06-30T07:09:48.483Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13721161vm-test-run-nixbot-gitlab> gitlab # [ 11.557356] gitaly[1372]: time="2026-06-30T07:09:48.484Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13721162vm-test-run-nixbot-gitlab> gitlab # [ 11.559199] gitaly[1372]: time="2026-06-30T07:09:48.484Z" level=info msg="Starting Gitaly" pid=1372 version=18.11.41163vm-test-run-nixbot-gitlab> gitlab # [ 11.756653] gitaly[1372]: time="2026-06-30T07:09:48.689Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=184 pid=13721164vm-test-run-nixbot-gitlab> gitlab # [ 11.757482] gitaly[1372]: time="2026-06-30T07:09:48.689Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13721165vm-test-run-nixbot-gitlab> gitlab # [ 11.758401] gitaly[1372]: time="2026-06-30T07:09:48.689Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13721166vm-test-run-nixbot-gitlab> gitlab # [ 11.759881] gitaly[1372]: time="2026-06-30T07:09:48.689Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1372/git-exec-3934216837.d/git pid=13721167vm-test-run-nixbot-gitlab> gitlab # [ 11.770425] gitaly[1372]: time="2026-06-30T07:09:48.702Z" level=info msg="finished detecting git version" duration_ms=13 pid=13721168vm-test-run-nixbot-gitlab> gitlab # [ 11.771231] gitaly[1372]: time="2026-06-30T07:09:48.702Z" level=info msg="using Git version" pid=1372 version=2.53.g1169vm-test-run-nixbot-gitlab> gitlab # [ 11.772389] gitaly[1372]: time="2026-06-30T07:09:48.704Z" level=info msg="clearing disk cache object folder" pid=1372 storage=default1170vm-test-run-nixbot-gitlab> gitlab # [ 11.772973] gitaly[1372]: time="2026-06-30T07:09:48.704Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache540446881 pid=1372 storage=default1171vm-test-run-nixbot-gitlab> gitlab # [ 11.774115] gitaly[1372]: time="2026-06-30T07:09:48.704Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache540446881 pid=1372 storage=default1172vm-test-run-nixbot-gitlab> gitlab # [ 11.775452] gitaly[1372]: time="2026-06-30T07:09:48.704Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13721173vm-test-run-nixbot-gitlab> gitlab # [ 11.776109] gitaly[1372]: time="2026-06-30T07:09:48.705Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache540446881 clear_duration_ms=0 pid=1372 storage=default1174vm-test-run-nixbot-gitlab> gitlab # [ 11.798830] gitaly[1372]: time="2026-06-30T07:09:48.705Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13721175vm-test-run-nixbot-gitlab> gitlab # [ 11.871149] gitaly[1372]: time="2026-06-30T07:09:48.803Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13721176vm-test-run-nixbot-gitlab> gitlab # [ 11.871457] gitaly[1372]: time="2026-06-30T07:09:48.803Z" level=info msg="counting repositories" pid=13721177vm-test-run-nixbot-gitlab> gitlab # [ 11.874203] gitaly[1372]: time="2026-06-30T07:09:48.803Z" level=info msg="starting to count repositories" pid=1372 storage_path=/var/gitlab/state/repositories1178vm-test-run-nixbot-gitlab> gitlab # [ 11.877737] gitaly[1372]: time="2026-06-30T07:09:48.804Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1372 storage_path=/var/gitlab/state/repositories1179vm-test-run-nixbot-gitlab> gitlab # [ 11.880962] gitaly[1372]: time="2026-06-30T07:09:48.804Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13721180vm-test-run-nixbot-gitlab> gitlab # [ 11.885975] gitaly[1372]: time="2026-06-30T07:09:48.804Z" level=info msg="finished tempdir cleaner walk" pid=1372 storage=default time_ms=01181vm-test-run-nixbot-gitlab> gitlab # [ 11.889322] gitaly[1372]: time="2026-06-30T07:09:48.804Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1372 schema=unix1182vm-test-run-nixbot-gitlab> gitlab # [ 11.891070] gitaly[1372]: time="2026-06-30T07:09:48.804Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1372/sock.d/intern pid=1372 schema=unix1183vm-test-run-nixbot-gitlab> gitlab # [ 11.926217] gitaly[1372]: time="2026-06-30T07:09:48.858Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13721184vm-test-run-nixbot-gitlab> gitlab # [ 11.927286] gitaly[1372]: time="2026-06-30T07:09:48.858Z" level=info msg="Started Gitaly" duration_ms=374 pid=13721185vm-test-run-nixbot-gitlab> gitlab # [ 11.928218] gitaly[1372]: time="2026-06-30T07:09:48.859Z" level=info msg="maintenance: daily scheduled" pid=1372 scheduled="2026-06-30 12:00:00 +0000 UTC"1186vm-test-run-nixbot-gitlab> gitlab # [ 12.278643] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1375]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1187vm-test-run-nixbot-gitlab> gitlab # [ 12.280265] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1375]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1188vm-test-run-nixbot-gitlab> gitlab # [ 12.327718] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1375]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1189vm-test-run-nixbot-gitlab> gitlab # [ 13.801156] gitaly[1372]: time="2026-06-30T07:09:50.733Z" level=info msg="License database preloaded" duration_ms=2248 pid=13721190vm-test-run-nixbot-gitlab> gitlab # [ 14.297750] dhcpcd[1225]: eth0: leased 10.0.2.15 for 86400 seconds1191vm-test-run-nixbot-gitlab> gitlab # [ 14.299290] dhcpcd[1225]: eth0: adding route to 10.0.2.0/241192vm-test-run-nixbot-gitlab> gitlab # [ 14.301939] dhcpcd[1225]: eth0: adding default route via 10.0.2.21193vm-test-run-nixbot-gitlab> gitlab # [ 14.422786] systemd[1]: Started DHCP Client.1194vm-test-run-nixbot-gitlab> gitlab # [ 14.426823] systemd[1]: Reached target Network is Online.1195vm-test-run-nixbot-gitlab> gitlab # [ 44.634186] postgres[1751]: [1751] ERROR: relation "feature_gates" does not exist at character 5231196vm-test-run-nixbot-gitlab> gitlab # [ 44.640515] postgres[1751]: [1751] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1197vm-test-run-nixbot-gitlab> gitlab # [ 44.645829] postgres[1751]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1198vm-test-run-nixbot-gitlab> gitlab # [ 44.651492] postgres[1751]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1199vm-test-run-nixbot-gitlab> gitlab # [ 44.656769] postgres[1751]: attidentity AS identity,1200vm-test-run-nixbot-gitlab> gitlab # [ 44.660786] postgres[1751]: attgenerated as attgenerated1201vm-test-run-nixbot-gitlab> gitlab # [ 44.664788] postgres[1751]: FROM pg_attribute a1202vm-test-run-nixbot-gitlab> gitlab # [ 44.668271] postgres[1751]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1203vm-test-run-nixbot-gitlab> gitlab # [ 44.673841] postgres[1751]: LEFT JOIN pg_type t ON a.atttypid = t.oid1204vm-test-run-nixbot-gitlab> gitlab # [ 44.678224] postgres[1751]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1205vm-test-run-nixbot-gitlab> gitlab # [ 44.684499] postgres[1751]: WHERE a.attrelid = '"feature_gates"'::regclass1206vm-test-run-nixbot-gitlab> gitlab # [ 44.688242] postgres[1751]: AND a.attnum > 0 AND NOT a.attisdropped1207vm-test-run-nixbot-gitlab> gitlab # [ 44.691710] postgres[1751]: ORDER BY a.attnum1208vm-test-run-nixbot-gitlab> gitlab # [ 44.694405] postgres[1751]:1209vm-test-run-nixbot-gitlab> gitlab # [ 44.875416] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1375]: W, [2026-06-30T07:10:21.806888 #1375] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1210vm-test-run-nixbot-gitlab> gitlab # [ 49.525810] gitlab-workhorse[1280]: time="2026-06-30T07:10:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1211vm-test-run-nixbot-gitlab> gitlab # [ 50.680956] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1815]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1212vm-test-run-nixbot-gitlab> gitlab # [ 50.689852] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1815]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1213vm-test-run-nixbot-gitlab> gitlab # [ 59.527890] gitlab-workhorse[1280]: time="2026-06-30T07:10:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1214vm-test-run-nixbot-gitlab> gitlab # [ 68.816960] postgres[1899]: [1899] ERROR: relation "table_without_models" does not exist1215vm-test-run-nixbot-gitlab> gitlab # [ 68.818670] postgres[1899]: [1899] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1216vm-test-run-nixbot-gitlab> gitlab # [ 69.529766] gitlab-workhorse[1280]: time="2026-06-30T07:10:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1217vm-test-run-nixbot-gitlab> gitlab # [ 79.540108] gitlab-workhorse[1280]: time="2026-06-30T07:10:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1218vm-test-run-nixbot-gitlab> gitlab # [ 84.572289] systemd[1]: Finished gitlab-db-config.service.1219vm-test-run-nixbot-gitlab> gitlab # [ 84.579930] systemd[1]: Started gitlab-sidekiq.service.1220vm-test-run-nixbot-gitlab> gitlab # [ 84.584321] systemd[1]: Starting gitlab.service...1221vm-test-run-nixbot-gitlab> gitlab # [ 84.927350] bundle[2111]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1222vm-test-run-nixbot-gitlab> gitlab # [ 84.929428] bundle[2111]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1223vm-test-run-nixbot-gitlab> gitlab # [ 84.932399] bundle[2111]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1224vm-test-run-nixbot-gitlab> gitlab # [ 87.968694] sidekiq-cluster[2110]: {"severity":"INFO","time":"2026-06-30T07:11:04.881Z","message":"Starting cluster with 1 processes"}1225vm-test-run-nixbot-gitlab> gitlab # [ 88.165294] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.072Z","pid":2111,"message":"* Enabling systemd notification integration"}1226vm-test-run-nixbot-gitlab> gitlab # [ 88.166460] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.096Z","pid":2111,"message":"Puma starting in cluster mode..."}1227vm-test-run-nixbot-gitlab> gitlab # [ 88.168274] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.097Z","pid":2111,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1228vm-test-run-nixbot-gitlab> gitlab # [ 88.169713] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.097Z","pid":2111,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1229vm-test-run-nixbot-gitlab> gitlab # [ 88.171361] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.097Z","pid":2111,"message":"* Min threads: 0"}1230vm-test-run-nixbot-gitlab> gitlab # [ 88.172541] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.097Z","pid":2111,"message":"* Max threads: 4"}1231vm-test-run-nixbot-gitlab> gitlab # [ 88.174171] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.097Z","pid":2111,"message":"* Environment: production"}1232vm-test-run-nixbot-gitlab> gitlab # [ 88.175303] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.097Z","pid":2111,"message":"* Master PID: 2111"}1233vm-test-run-nixbot-gitlab> gitlab # [ 88.177126] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.097Z","pid":2111,"message":"* Workers: 2"}1234vm-test-run-nixbot-gitlab> gitlab # [ 88.178339] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.097Z","pid":2111,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1235vm-test-run-nixbot-gitlab> gitlab # [ 88.179988] bundle[2111]: {"timestamp":"2026-06-30T07:11:05.097Z","pid":2111,"message":"* Preloading application"}1236vm-test-run-nixbot-gitlab> gitlab # [ 89.551922] gitlab-workhorse[1280]: time="2026-06-30T07:11:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1237vm-test-run-nixbot-gitlab> gitlab # [ 99.561759] gitlab-workhorse[1280]: time="2026-06-30T07:11:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1238vm-test-run-nixbot-gitlab> gitlab # [ 109.570541] gitlab-workhorse[1280]: time="2026-06-30T07:11:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1239vm-test-run-nixbot-gitlab> gitlab # [ 113.859257] bundle[2111]: W, [2026-06-30T07:11:30.790813 #2111] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1240vm-test-run-nixbot-gitlab> gitlab # [ 116.996482] sidekiq-cluster[2141]: 2026-06-30T07:11:33.927Z pid=2141 tid=tzp INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2141, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>428826624}1241vm-test-run-nixbot-gitlab> gitlab # [ 117.196570] sidekiq-cluster[2141]: W, [2026-06-30T07:11:34.128888 #2141] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1242vm-test-run-nixbot-gitlab> gitlab # [ 117.239379] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:34.171Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1243vm-test-run-nixbot-gitlab> gitlab # [ 119.295583] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.227Z","message":"GitLab reliable fetch activated!"}1244vm-test-run-nixbot-gitlab> gitlab # [ 119.297210] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.227Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1245vm-test-run-nixbot-gitlab> gitlab # [ 119.326690] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.259Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1246vm-test-run-nixbot-gitlab> gitlab # [ 119.329965] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.262Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1247vm-test-run-nixbot-gitlab> gitlab # [ 119.334815] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.267Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1248vm-test-run-nixbot-gitlab> gitlab # [ 119.340559] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.272Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1249vm-test-run-nixbot-gitlab> gitlab # [ 119.347857] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.280Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1250vm-test-run-nixbot-gitlab> gitlab # [ 119.355252] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.287Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1251vm-test-run-nixbot-gitlab> gitlab # [ 119.362417] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.294Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1252vm-test-run-nixbot-gitlab> gitlab # [ 119.369459] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.301Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1253vm-test-run-nixbot-gitlab> gitlab # [ 119.375242] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.307Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [ 119.380842] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.313Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [ 119.386450] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.318Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [ 119.392117] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.324Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [ 119.397322] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.329Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [ 119.401706] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.334Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [ 119.405479] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.337Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [ 119.409292] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.340Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [ 119.413354] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.343Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [ 119.414382] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.346Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [ 119.422933] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.355Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [ 119.429324] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.361Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [ 119.435604] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.368Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [ 119.442854] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.375Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [ 119.447211] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.378Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [ 119.448988] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.381Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [ 119.452553] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.385Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [ 119.456049] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.388Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [ 119.459788] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.392Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [ 119.469496] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.401Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [ 119.476866] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.409Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [ 119.483683] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.416Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [ 119.490349] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.422Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [ 119.496368] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.428Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [ 119.499685] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.432Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [ 119.503413] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.435Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [ 119.506969] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.439Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [ 119.510199] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.442Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [ 119.513476] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.446Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [ 119.522724] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.455Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [ 119.529678] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.462Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [ 119.535661] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.468Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1285vm-test-run-nixbot-gitlab> gitlab # [ 119.542204] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.474Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1286vm-test-run-nixbot-gitlab> gitlab # [ 119.548469] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.480Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1287vm-test-run-nixbot-gitlab> gitlab # [ 119.554319] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.486Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1288vm-test-run-nixbot-gitlab> gitlab # [ 119.560997] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.493Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1289vm-test-run-nixbot-gitlab> gitlab # [ 119.568172] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.500Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1290vm-test-run-nixbot-gitlab> gitlab # [ 119.575955] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.508Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1291vm-test-run-nixbot-gitlab> gitlab # [ 119.580263] gitlab-workhorse[1280]: time="2026-06-30T07:11:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1292vm-test-run-nixbot-gitlab> gitlab # [ 119.587441] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.519Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1293vm-test-run-nixbot-gitlab> gitlab # [ 119.593951] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.526Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1294vm-test-run-nixbot-gitlab> gitlab # [ 119.600972] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.533Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1295vm-test-run-nixbot-gitlab> gitlab # [ 119.608158] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.540Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1296vm-test-run-nixbot-gitlab> gitlab # [ 119.616142] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.548Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1297vm-test-run-nixbot-gitlab> gitlab # [ 119.623612] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.556Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1298vm-test-run-nixbot-gitlab> gitlab # [ 119.627588] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.560Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1299vm-test-run-nixbot-gitlab> gitlab # [ 119.640104] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.572Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1300vm-test-run-nixbot-gitlab> gitlab # [ 119.644281] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.576Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1301vm-test-run-nixbot-gitlab> gitlab # [ 119.648920] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.580Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1302vm-test-run-nixbot-gitlab> gitlab # [ 119.654917] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.587Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1303vm-test-run-nixbot-gitlab> gitlab # [ 119.662258] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.594Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1304vm-test-run-nixbot-gitlab> gitlab # [ 119.666612] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.598Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1305vm-test-run-nixbot-gitlab> gitlab # [ 119.672571] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.605Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1306vm-test-run-nixbot-gitlab> gitlab # [ 119.677151] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.609Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1307vm-test-run-nixbot-gitlab> gitlab # [ 119.682276] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.614Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1308vm-test-run-nixbot-gitlab> gitlab # [ 119.687947] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.620Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1309vm-test-run-nixbot-gitlab> gitlab # [ 119.696140] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.628Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1310vm-test-run-nixbot-gitlab> gitlab # [ 119.703636] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.636Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1311vm-test-run-nixbot-gitlab> gitlab # [ 119.708316] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.639Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1312vm-test-run-nixbot-gitlab> gitlab # [ 119.714207] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.646Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1313vm-test-run-nixbot-gitlab> gitlab # [ 119.723162] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.655Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1314vm-test-run-nixbot-gitlab> gitlab # [ 119.764076] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.696Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1315vm-test-run-nixbot-gitlab> gitlab # [ 119.776537] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.708Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1316vm-test-run-nixbot-gitlab> gitlab # [ 119.788875] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.721Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1317vm-test-run-nixbot-gitlab> gitlab # [ 119.801718] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.733Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1318vm-test-run-nixbot-gitlab> gitlab # [ 119.816570] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.748Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1319vm-test-run-nixbot-gitlab> gitlab # [ 119.830224] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.762Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1320vm-test-run-nixbot-gitlab> gitlab # [ 119.846184] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.778Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1321vm-test-run-nixbot-gitlab> gitlab # [ 119.859868] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.792Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1322vm-test-run-nixbot-gitlab> gitlab # [ 119.873567] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.805Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1323vm-test-run-nixbot-gitlab> gitlab # [ 119.884448] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.816Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1324vm-test-run-nixbot-gitlab> gitlab # [ 119.893526] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.825Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1325vm-test-run-nixbot-gitlab> gitlab # [ 119.902602] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.834Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1326vm-test-run-nixbot-gitlab> gitlab # [ 119.910597] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.842Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1327vm-test-run-nixbot-gitlab> gitlab # [ 119.920184] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.852Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1328vm-test-run-nixbot-gitlab> gitlab # [ 119.928597] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.860Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1329vm-test-run-nixbot-gitlab> gitlab # [ 119.939531] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.871Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1330vm-test-run-nixbot-gitlab> gitlab # [ 119.947908] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.880Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1331vm-test-run-nixbot-gitlab> gitlab # [ 119.957871] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.890Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1332vm-test-run-nixbot-gitlab> gitlab # [ 119.966184] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.898Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1333vm-test-run-nixbot-gitlab> gitlab # [ 119.976893] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.909Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1334vm-test-run-nixbot-gitlab> gitlab # [ 119.985620] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.917Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1335vm-test-run-nixbot-gitlab> gitlab # [ 119.995195] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.927Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1336vm-test-run-nixbot-gitlab> gitlab # [ 120.003598] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.935Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1337vm-test-run-nixbot-gitlab> gitlab # [ 120.013362] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.945Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1338vm-test-run-nixbot-gitlab> gitlab # [ 120.021605] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.953Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1339vm-test-run-nixbot-gitlab> gitlab # [ 120.030232] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.962Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1340vm-test-run-nixbot-gitlab> gitlab # [ 120.040592] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.972Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1341vm-test-run-nixbot-gitlab> gitlab # [ 120.049984] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.982Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1342vm-test-run-nixbot-gitlab> gitlab # [ 120.060779] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:36.992Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1343vm-test-run-nixbot-gitlab> gitlab # [ 120.069157] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.001Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1344vm-test-run-nixbot-gitlab> gitlab # [ 120.078372] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.010Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1345vm-test-run-nixbot-gitlab> gitlab # [ 120.104246] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.036Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1346vm-test-run-nixbot-gitlab> gitlab # [ 120.110871] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.043Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1347vm-test-run-nixbot-gitlab> gitlab # [ 120.116849] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.049Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1348vm-test-run-nixbot-gitlab> gitlab # [ 120.126547] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.058Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1349vm-test-run-nixbot-gitlab> gitlab # [ 120.134727] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.067Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1350vm-test-run-nixbot-gitlab> gitlab # [ 120.142699] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.075Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1351vm-test-run-nixbot-gitlab> gitlab # [ 120.147470] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.080Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1352vm-test-run-nixbot-gitlab> gitlab # [ 120.152280] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.083Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1353vm-test-run-nixbot-gitlab> gitlab # [ 120.157431] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.089Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1354vm-test-run-nixbot-gitlab> gitlab # [ 120.164590] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.096Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1355vm-test-run-nixbot-gitlab> gitlab # [ 120.171582] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:11:37.104Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1356vm-test-run-nixbot-gitlab> gitlab # [ 122.748482] sidekiq-cluster[2141]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics'1357vm-test-run-nixbot-gitlab> gitlab # [ 122.754079] sidekiq-cluster[2141]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1358vm-test-run-nixbot-gitlab> gitlab # [ 122.756666] sidekiq-cluster[2141]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1359vm-test-run-nixbot-gitlab> gitlab # [ 122.758889] sidekiq-cluster[2141]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1360vm-test-run-nixbot-gitlab> gitlab # [ 122.761260] sidekiq-cluster[2141]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1361vm-test-run-nixbot-gitlab> gitlab # [ 122.763402] sidekiq-cluster[2141]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1362vm-test-run-nixbot-gitlab> gitlab # [ 122.765911] sidekiq-cluster[2141]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1363vm-test-run-nixbot-gitlab> gitlab # [ 129.581621] gitlab-workhorse[1280]: time="2026-06-30T07:11:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1364vm-test-run-nixbot-gitlab> gitlab # [ 139.582224] gitlab-workhorse[1280]: time="2026-06-30T07:11:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1365vm-test-run-nixbot-gitlab> gitlab # [ 144.548281] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:01.480Z","message":"Booted Rails 7.2.3 application in production environment"}1366vm-test-run-nixbot-gitlab> gitlab # [ 144.549372] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:01.481Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1367vm-test-run-nixbot-gitlab> gitlab # [ 144.552067] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:01.482Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1368vm-test-run-nixbot-gitlab> gitlab # [ 144.554628] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:01.482Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1369vm-test-run-nixbot-gitlab> gitlab # [ 144.557946] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:01.484Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1370vm-test-run-nixbot-gitlab> gitlab # [ 144.614587] bundle[2111]: {"timestamp":"2026-06-30T07:12:01.546Z","pid":2111,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1371vm-test-run-nixbot-gitlab> gitlab # [ 144.620833] bundle[2111]: {"timestamp":"2026-06-30T07:12:01.546Z","pid":2111,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1372vm-test-run-nixbot-gitlab> gitlab # [ 144.621141] bundle[2111]: {"timestamp":"2026-06-30T07:12:01.546Z","pid":2111,"message":"! #\u003cThread:0x000074533b402150 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1373vm-test-run-nixbot-gitlab> gitlab # [ 144.628795] bundle[2111]: {"timestamp":"2026-06-30T07:12:01.546Z","pid":2111,"message":"! #\u003cThread:0x000074533b65b8e8 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1374vm-test-run-nixbot-gitlab> gitlab # [ 144.631589] bundle[2111]: {"timestamp":"2026-06-30T07:12:01.546Z","pid":2111,"message":"Use Ctrl-C to stop"}1375vm-test-run-nixbot-gitlab> gitlab # [ 144.633670] bundle[2111]: {"timestamp":"2026-06-30T07:12:01.547Z","pid":2111,"message":"Sending status to systemd every 1.0 sec"}1376vm-test-run-nixbot-gitlab> gitlab # [ 145.077143] bundle[2111]: {"timestamp":"2026-06-30T07:12:02.008Z","pid":2111,"message":"- Worker 1 (PID: 2684) booted in 0.37s, phase: 0"}1377vm-test-run-nixbot-gitlab> gitlab # [ 145.098371] bundle[2111]: {"timestamp":"2026-06-30T07:12:02.030Z","pid":2111,"message":"- Worker 0 (PID: 2680) booted in 0.42s, phase: 0"}1378vm-test-run-nixbot-gitlab> gitlab # [ 145.103604] systemd[1]: Started gitlab.service.1379vm-test-run-nixbot-gitlab> gitlab # [ 145.105243] systemd[1]: Reached target Common target for all GitLab services..1380vm-test-run-nixbot-gitlab> gitlab # [ 145.108993] systemd[1]: Starting setup-gitlab.service...1381vm-test-run-nixbot-gitlab> gitlab # [ 145.127390] setup-gitlab-start[2717]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1382vm-test-run-nixbot-gitlab> gitlab # [ 146.412742] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:03.344Z","message":"Cleaning working queues"}1383vm-test-run-nixbot-gitlab> gitlab # [ 146.414825] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:03.346Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2f9a27e7634e85673333f07d","created_at":"2026-06-30T07:11:00.806Z","trace_propagation_headers":{"sentry-trace":"e29cb4448a2d4e269e07d392241dcdb9-2e18c3bc51fe4a57","baggage":"sentry-trace_id=e29cb4448a2d4e269e07d392241dcdb9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"675b9d9fb294bb60a2466696520075fe","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-30T07:11:00.816Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2f9a27e7634e85673333f07d: start","job_status":"start","queue_duration_s":62.529646,"scheduling_latency_s":62.529707}1384vm-test-run-nixbot-gitlab> gitlab # [ 146.671225] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:03.602Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2f9a27e7634e85673333f07d","created_at":"2026-06-30T07:11:00.806Z","trace_propagation_headers":{"sentry-trace":"e29cb4448a2d4e269e07d392241dcdb9-2e18c3bc51fe4a57","baggage":"sentry-trace_id=e29cb4448a2d4e269e07d392241dcdb9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"675b9d9fb294bb60a2466696520075fe","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-30T07:11:00.816Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2f9a27e7634e85673333f07d: done: 0.256542 sec","job_status":"done","queue_duration_s":62.529646,"scheduling_latency_s":62.529707,"redis_calls":24,"redis_duration_s":0.010439,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.004535,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003721,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002183,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.118767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.155067753,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.256542,"completed_at":"2026-06-30T07:12:03.602Z","load_balancing_strategy":"primary","db_duration_s":0.099696,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1385vm-test-run-nixbot-gitlab> gitlab # [ 147.685971] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:12:04 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 25341386vm-test-run-nixbot-gitlab> gitlab # [ 147.695564] setup-gitlab-start[2743]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1387vm-test-run-nixbot-gitlab> gitlab # [ 147.698932] setup-gitlab-start[2744]: ++ jq -r .access_token1388vm-test-run-nixbot-gitlab> gitlab # [ 149.071551] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:12:06 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 13411389vm-test-run-nixbot-gitlab> gitlab # [ 149.082843] setup-gitlab-start[2717]: + BEARER=8474ca25eb314ff09be742be92b15190486076877f30c008c1ba20e4b26e4ece1390vm-test-run-nixbot-gitlab> gitlab # [ 149.084212] setup-gitlab-start[2717]: + AUTH='Authorization: Bearer 8474ca25eb314ff09be742be92b15190486076877f30c008c1ba20e4b26e4ece'1391vm-test-run-nixbot-gitlab> gitlab # [ 149.086900] setup-gitlab-start[2717]: + curl -fs -X PUT -H 'Authorization: Bearer 8474ca25eb314ff09be742be92b15190486076877f30c008c1ba20e4b26e4ece' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1392vm-test-run-nixbot-gitlab> gitlab # [ 149.911362] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:12:06 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 7991393vm-test-run-nixbot-gitlab> gitlab # [ 149.923973] setup-gitlab-start[2717]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1394vm-test-run-nixbot-gitlab> gitlab # [ 149.949122] runuser[2768]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1395vm-test-run-nixbot-gitlab> gitlab # [ 150.309847] setup-gitlab-start[2769]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1396vm-test-run-nixbot-gitlab> gitlab # [ 150.311207] setup-gitlab-start[2769]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1397vm-test-run-nixbot-gitlab> gitlab # [ 150.313978] setup-gitlab-start[2769]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1398vm-test-run-nixbot-gitlab> gitlab # [ 150.900405] setup-gitlab-start[2769]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1399vm-test-run-nixbot-gitlab> gitlab # [ 150.901915] setup-gitlab-start[2769]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1400vm-test-run-nixbot-gitlab> gitlab # [ 150.905235] setup-gitlab-start[2769]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1401vm-test-run-nixbot-gitlab> gitlab # [ 157.767531] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:14.698Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"74a8c47b1408caaaee516504","created_at":"2026-06-30T07:12:14.667Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8208ac785bd4cddd6a99baedaf6a5b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:14.695Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-74a8c47b1408caaaee516504: start","job_status":"start","queue_duration_s":0.00333,"scheduling_latency_s":0.003367}1402vm-test-run-nixbot-gitlab> gitlab # [ 157.925439] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:14.857Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1403vm-test-run-nixbot-gitlab> gitlab # [ 157.955946] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:14.887Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"74a8c47b1408caaaee516504","created_at":"2026-06-30T07:12:14.667Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8208ac785bd4cddd6a99baedaf6a5b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:14.695Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-74a8c47b1408caaaee516504: done: 0.188897 sec","job_status":"done","queue_duration_s":0.00333,"scheduling_latency_s":0.003367,"redis_calls":10,"redis_duration_s":0.035614,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.01532,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012494,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0078,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003886248,"duration_s":0.188897,"completed_at":"2026-06-30T07:12:14.887Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404vm-test-run-nixbot-gitlab> gitlab # [ 158.046352] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:14.891Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ff3d650cce824eae08f59b70","created_at":"2026-06-30T07:12:14.798Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a87191fa4188f7fbdb47848857319d81","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:14.824Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ff3d650cce824eae08f59b70: start","job_status":"start","queue_duration_s":0.06713,"scheduling_latency_s":0.0673}1405vm-test-run-nixbot-gitlab> gitlab # [ 158.799201] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:15.730Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ff3d650cce824eae08f59b70","created_at":"2026-06-30T07:12:14.798Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a87191fa4188f7fbdb47848857319d81","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:14.824Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ff3d650cce824eae08f59b70: done: 0.838679 sec","job_status":"done","queue_duration_s":0.06713,"scheduling_latency_s":0.0673,"redis_calls":14,"redis_duration_s":0.005519,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000938,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002392,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002189,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.387566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000539175,"exclusive_lock_hold_duration_s":0.806422603,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.838679,"completed_at":"2026-06-30T07:12:15.730Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.09434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406vm-test-run-nixbot-gitlab> gitlab # [ 158.869514] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:15.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"e13884bae29327e2205228af","created_at":"2026-06-30T07:12:14.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.023Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-e13884bae29327e2205228af: start","job_status":"start","queue_duration_s":0.777771,"scheduling_latency_s":0.777815}1407vm-test-run-nixbot-gitlab> gitlab # [ 158.963451] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:15.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"e13884bae29327e2205228af","created_at":"2026-06-30T07:12:14.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.023Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-e13884bae29327e2205228af: done: 0.09366 sec","job_status":"done","queue_duration_s":0.777771,"scheduling_latency_s":0.777815,"redis_calls":19,"redis_duration_s":0.008726000000000001,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001546,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000711,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004719,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00175,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000816305,"duration_s":0.09366,"completed_at":"2026-06-30T07:12:15.894Z","load_balancing_strategy":"primary","db_duration_s":0.004565,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408vm-test-run-nixbot-gitlab> gitlab # [ 159.043343] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:15.975Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d4e564f9ffafae2d1e1dda8d","created_at":"2026-06-30T07:12:15.034Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5604ae2f5ed60472b05004b97fc67253","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.047Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d4e564f9ffafae2d1e1dda8d: start","job_status":"start","queue_duration_s":0.928184,"scheduling_latency_s":0.92823}1409vm-test-run-nixbot-gitlab> gitlab # [ 159.090226] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.022Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d4e564f9ffafae2d1e1dda8d","created_at":"2026-06-30T07:12:15.034Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5604ae2f5ed60472b05004b97fc67253","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.047Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d4e564f9ffafae2d1e1dda8d: done: 0.046992 sec","job_status":"done","queue_duration_s":0.928184,"scheduling_latency_s":0.92823,"redis_calls":16,"redis_duration_s":0.007311,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003572,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002882,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000857,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054085,"duration_s":0.046992,"completed_at":"2026-06-30T07:12:16.022Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410vm-test-run-nixbot-gitlab> gitlab # [ 159.160260] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"50c148370d26ceb66ce6fd3a","created_at":"2026-06-30T07:12:15.060Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93106c50d48fa17827b4e3c7537020c3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.073Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-50c148370d26ceb66ce6fd3a: start","job_status":"start","queue_duration_s":1.018661,"scheduling_latency_s":1.018737}1411vm-test-run-nixbot-gitlab> gitlab # [ 159.220960] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.152Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"50c148370d26ceb66ce6fd3a","created_at":"2026-06-30T07:12:15.060Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93106c50d48fa17827b4e3c7537020c3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.073Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-50c148370d26ceb66ce6fd3a: done: 0.060985 sec","job_status":"done","queue_duration_s":1.018661,"scheduling_latency_s":1.018737,"redis_calls":10,"redis_duration_s":0.0064280000000000006,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001177,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001384,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001005714,"duration_s":0.060985,"completed_at":"2026-06-30T07:12:16.152Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412vm-test-run-nixbot-gitlab> gitlab # [ 159.292977] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.224Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1ef8f0a06d06e8a610f365bb","created_at":"2026-06-30T07:12:15.080Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d5ab66b2aa5f6b5e0db2a26d104204c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.090Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1ef8f0a06d06e8a610f365bb: start","job_status":"start","queue_duration_s":1.13335,"scheduling_latency_s":1.13346}1413vm-test-run-nixbot-gitlab> gitlab # [ 159.347428] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.279Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1ef8f0a06d06e8a610f365bb","created_at":"2026-06-30T07:12:15.080Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d5ab66b2aa5f6b5e0db2a26d104204c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.090Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1ef8f0a06d06e8a610f365bb: done: 0.054982 sec","job_status":"done","queue_duration_s":1.13335,"scheduling_latency_s":1.13346,"redis_calls":12,"redis_duration_s":0.008222,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001366,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001422,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004023,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001052089,"duration_s":0.054982,"completed_at":"2026-06-30T07:12:16.278Z","load_balancing_strategy":"primary","db_duration_s":0.001166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414vm-test-run-nixbot-gitlab> gitlab # [ 159.417241] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"ded37fc200da01d0baa6cf60","created_at":"2026-06-30T07:12:15.096Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03be4d05279632ee96338efa96da91e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.107Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-ded37fc200da01d0baa6cf60: start","job_status":"start","queue_duration_s":1.241873,"scheduling_latency_s":1.241946}1415vm-test-run-nixbot-gitlab> gitlab # [ 159.475640] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.407Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"ded37fc200da01d0baa6cf60","created_at":"2026-06-30T07:12:15.096Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03be4d05279632ee96338efa96da91e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.107Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-ded37fc200da01d0baa6cf60: done: 0.058529 sec","job_status":"done","queue_duration_s":1.241873,"scheduling_latency_s":1.241946,"redis_calls":12,"redis_duration_s":0.007567,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002599,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001005156,"duration_s":0.058529,"completed_at":"2026-06-30T07:12:16.407Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416vm-test-run-nixbot-gitlab> gitlab # [ 159.547254] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.478Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"eda4b01fd22213ff78581f66","created_at":"2026-06-30T07:12:15.115Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae52bcd41d95f0fe5c8451249ecc0ec1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.120Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-eda4b01fd22213ff78581f66: start","job_status":"start","queue_duration_s":1.357538,"scheduling_latency_s":1.357612}1417vm-test-run-nixbot-gitlab> gitlab # [ 159.596738] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"eda4b01fd22213ff78581f66","created_at":"2026-06-30T07:12:15.115Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae52bcd41d95f0fe5c8451249ecc0ec1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.120Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-eda4b01fd22213ff78581f66: done: 0.050227 sec","job_status":"done","queue_duration_s":1.357538,"scheduling_latency_s":1.357612,"redis_calls":10,"redis_duration_s":0.00649,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001429,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003795,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001266,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00093727,"duration_s":0.050227,"completed_at":"2026-06-30T07:12:16.528Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418vm-test-run-nixbot-gitlab> gitlab # [ 159.669449] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.598Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ce92e8c84b818b49ae33c0e4","created_at":"2026-06-30T07:12:15.131Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"bcb6a64e431350fcb98d44212a05a80c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.138Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ce92e8c84b818b49ae33c0e4: start","job_status":"start","queue_duration_s":1.460024,"scheduling_latency_s":1.460069}1419vm-test-run-nixbot-gitlab> gitlab # [ 159.689528] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.621Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ce92e8c84b818b49ae33c0e4","created_at":"2026-06-30T07:12:15.131Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"bcb6a64e431350fcb98d44212a05a80c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.138Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ce92e8c84b818b49ae33c0e4: done: 0.023171 sec","job_status":"done","queue_duration_s":1.460024,"scheduling_latency_s":1.460069,"redis_calls":10,"redis_duration_s":0.0025,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000475,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001576,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000425753,"duration_s":0.023171,"completed_at":"2026-06-30T07:12:16.621Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001303,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420vm-test-run-nixbot-gitlab> gitlab # [ 159.691780] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"40fa5e5f53802f1d0471a6bf","created_at":"2026-06-30T07:12:15.143Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d35ae212c89ee28454b3c2ab165917ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.149Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-40fa5e5f53802f1d0471a6bf: start","job_status":"start","queue_duration_s":1.473562,"scheduling_latency_s":1.473584}1421vm-test-run-nixbot-gitlab> gitlab # [ 159.712568] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.636Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1422vm-test-run-nixbot-gitlab> gitlab # [ 159.717967] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"40fa5e5f53802f1d0471a6bf","created_at":"2026-06-30T07:12:15.143Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d35ae212c89ee28454b3c2ab165917ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.149Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-40fa5e5f53802f1d0471a6bf: done: 0.023989 sec","job_status":"done","queue_duration_s":1.473562,"scheduling_latency_s":1.473584,"redis_calls":10,"redis_duration_s":0.003127,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000614,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000757918,"duration_s":0.023989,"completed_at":"2026-06-30T07:12:16.647Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423vm-test-run-nixbot-gitlab> gitlab # [ 159.786492] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.648Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7bd29ed84286cd7abd374e73","created_at":"2026-06-30T07:12:15.154Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6419005ca694967925e13c4224c8e465","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.161Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7bd29ed84286cd7abd374e73: start","job_status":"start","queue_duration_s":1.487321,"scheduling_latency_s":1.487344}1424vm-test-run-nixbot-gitlab> gitlab # [ 159.866812] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.661Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1425vm-test-run-nixbot-gitlab> gitlab # [ 159.869262] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7bd29ed84286cd7abd374e73","created_at":"2026-06-30T07:12:15.154Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6419005ca694967925e13c4224c8e465","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.161Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7bd29ed84286cd7abd374e73: done: 0.01895 sec","job_status":"done","queue_duration_s":1.487321,"scheduling_latency_s":1.487344,"redis_calls":10,"redis_duration_s":0.0026290000000000003,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000455,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001685,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000403962,"duration_s":0.01895,"completed_at":"2026-06-30T07:12:16.667Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426vm-test-run-nixbot-gitlab> gitlab # [ 159.883972] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c0ef5f5c61dd340b64e24a41","created_at":"2026-06-30T07:12:15.171Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c3ecf61d181f1cb778995d253491bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.180Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c0ef5f5c61dd340b64e24a41: start","job_status":"start","queue_duration_s":1.489006,"scheduling_latency_s":1.489028}1427vm-test-run-nixbot-gitlab> gitlab # [ 159.966907] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.706Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c0ef5f5c61dd340b64e24a41","created_at":"2026-06-30T07:12:15.171Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c3ecf61d181f1cb778995d253491bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.180Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c0ef5f5c61dd340b64e24a41: done: 0.037034 sec","job_status":"done","queue_duration_s":1.489006,"scheduling_latency_s":1.489028,"redis_calls":13,"redis_duration_s":0.0034579999999999997,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000453,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001787,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.014568891,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.037034,"completed_at":"2026-06-30T07:12:16.706Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006658,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1428vm-test-run-nixbot-gitlab> gitlab # [ 159.977220] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:16.708Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"30d2ca95371b176815aebfe9","created_at":"2026-06-30T07:12:15.188Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fee5f077e4988bfda42ea097aa7f6aa6","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.195Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-30d2ca95371b176815aebfe9: start","job_status":"start","queue_duration_s":1.512774,"scheduling_latency_s":1.512798}1429vm-test-run-nixbot-gitlab> gitlab # [ 160.223732] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"30d2ca95371b176815aebfe9","created_at":"2026-06-30T07:12:15.188Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fee5f077e4988bfda42ea097aa7f6aa6","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.195Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-30d2ca95371b176815aebfe9: done: 0.447431 sec","job_status":"done","queue_duration_s":1.512774,"scheduling_latency_s":1.512798,"redis_calls":574,"redis_duration_s":0.14937,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001854,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.147081,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.337658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000527162,"duration_s":0.447431,"completed_at":"2026-06-30T07:12:17.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430vm-test-run-nixbot-gitlab> gitlab # [ 160.227297] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.157Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e2678f3f14e0517736bb3a5f","created_at":"2026-06-30T07:12:15.202Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"324a2eccf4a80f0506a729f993c1aedb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.208Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e2678f3f14e0517736bb3a5f: start","job_status":"start","queue_duration_s":1.948425,"scheduling_latency_s":1.948448}1431vm-test-run-nixbot-gitlab> gitlab # [ 160.406851] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e2678f3f14e0517736bb3a5f","created_at":"2026-06-30T07:12:15.202Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"324a2eccf4a80f0506a729f993c1aedb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.208Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e2678f3f14e0517736bb3a5f: done: 0.180824 sec","job_status":"done","queue_duration_s":1.948425,"scheduling_latency_s":1.948448,"redis_calls":14,"redis_duration_s":0.005812,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000999,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002112,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000534705,"exclusive_lock_hold_duration_s":0.14897507999999998,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.180824,"completed_at":"2026-06-30T07:12:17.338Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.127884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432vm-test-run-nixbot-gitlab> gitlab # [ 160.411187] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4ae45b00178dce951c8a9427","created_at":"2026-06-30T07:12:15.215Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef6c6cbbcdcdd15bc47458681df37512","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.221Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4ae45b00178dce951c8a9427: start","job_status":"start","queue_duration_s":2.120215,"scheduling_latency_s":2.120259}1433vm-test-run-nixbot-gitlab> gitlab # [ 160.445140] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4ae45b00178dce951c8a9427","created_at":"2026-06-30T07:12:15.215Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef6c6cbbcdcdd15bc47458681df37512","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.221Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4ae45b00178dce951c8a9427: done: 0.025841 sec","job_status":"done","queue_duration_s":2.120215,"scheduling_latency_s":2.120259,"redis_calls":12,"redis_duration_s":0.003143,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000625,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001431,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000532749,"exclusive_lock_hold_duration_s":0.0045924830000000005,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.025841,"completed_at":"2026-06-30T07:12:17.367Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002741,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434vm-test-run-nixbot-gitlab> gitlab # [ 160.514433] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"3975bf1c80d96ae7564deccf","created_at":"2026-06-30T07:12:15.226Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47b0bf612ad2957cbaa7b4e8584e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.233Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-3975bf1c80d96ae7564deccf: start","job_status":"start","queue_duration_s":2.135013,"scheduling_latency_s":2.135034}1435vm-test-run-nixbot-gitlab> gitlab # [ 160.537397] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"3975bf1c80d96ae7564deccf","created_at":"2026-06-30T07:12:15.226Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47b0bf612ad2957cbaa7b4e8584e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.233Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-3975bf1c80d96ae7564deccf: done: 0.019882 sec","job_status":"done","queue_duration_s":2.135013,"scheduling_latency_s":2.135034,"redis_calls":10,"redis_duration_s":0.002461,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001417,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000517,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000351162,"duration_s":0.019882,"completed_at":"2026-06-30T07:12:17.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436vm-test-run-nixbot-gitlab> gitlab # [ 160.630796] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4baa96668a330d345dcfb708","created_at":"2026-06-30T07:12:15.238Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c974c3f36a7a4a5803998e5e7fc27111","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.245Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4baa96668a330d345dcfb708: start","job_status":"start","queue_duration_s":2.144642,"scheduling_latency_s":2.144661}1437vm-test-run-nixbot-gitlab> gitlab # [ 160.632460] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.417Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4baa96668a330d345dcfb708","created_at":"2026-06-30T07:12:15.238Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c974c3f36a7a4a5803998e5e7fc27111","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.245Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4baa96668a330d345dcfb708: done: 0.026949 sec","job_status":"done","queue_duration_s":2.144642,"scheduling_latency_s":2.144661,"redis_calls":10,"redis_duration_s":0.002424,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000504,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001393,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000362895,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.026949,"completed_at":"2026-06-30T07:12:17.417Z","load_balancing_strategy":"primary","db_duration_s":0.008394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438vm-test-run-nixbot-gitlab> gitlab # [ 160.743552] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"405658dd263cfca237a17932","created_at":"2026-06-30T07:12:15.250Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5efcdcde03c072f9e68c957ddfefa37","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-30T07:12:15.257Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-405658dd263cfca237a17932: start","job_status":"start","queue_duration_s":2.161074,"scheduling_latency_s":2.161096}1439vm-test-run-nixbot-gitlab> gitlab # [ 160.745411] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.429Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1440vm-test-run-nixbot-gitlab> gitlab # [ 160.747346] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"405658dd263cfca237a17932","created_at":"2026-06-30T07:12:15.250Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5efcdcde03c072f9e68c957ddfefa37","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-30T07:12:15.257Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-405658dd263cfca237a17932: done: 0.086058 sec","job_status":"done","queue_duration_s":2.161074,"scheduling_latency_s":2.161096,"redis_calls":10,"redis_duration_s":0.002316,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000379937,"duration_s":0.086058,"completed_at":"2026-06-30T07:12:17.504Z","load_balancing_strategy":"primary","db_duration_s":0.000411,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441vm-test-run-nixbot-gitlab> gitlab # [ 160.789156] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.506Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"96c2e4176741e8fb7293fb05","created_at":"2026-06-30T07:12:15.265Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.273Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-96c2e4176741e8fb7293fb05: start","job_status":"start","queue_duration_s":2.233253,"scheduling_latency_s":2.233284}1442vm-test-run-nixbot-gitlab> gitlab # [ 160.811241] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.602Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"96c2e4176741e8fb7293fb05","created_at":"2026-06-30T07:12:15.265Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.273Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-96c2e4176741e8fb7293fb05: done: 0.095686 sec","job_status":"done","queue_duration_s":2.233253,"scheduling_latency_s":2.233284,"redis_calls":71,"redis_duration_s":0.016994000000000002,"redis_read_bytes":120,"redis_write_bytes":42574,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002341,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000863,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013093,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000697,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000415136,"duration_s":0.095686,"completed_at":"2026-06-30T07:12:17.602Z","load_balancing_strategy":"primary","db_duration_s":0.002007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443vm-test-run-nixbot-gitlab> gitlab # [ 160.916410] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.603Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"24cc52a3df06bcd173e8eec5","created_at":"2026-06-30T07:12:15.280Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b08f7f6f4fb6dccbdfb1db0dc68d5340","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.287Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-24cc52a3df06bcd173e8eec5: start","job_status":"start","queue_duration_s":2.3158,"scheduling_latency_s":2.315823}1444vm-test-run-nixbot-gitlab> gitlab # [ 160.917965] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.615Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1445vm-test-run-nixbot-gitlab> gitlab # [ 160.919641] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"24cc52a3df06bcd173e8eec5","created_at":"2026-06-30T07:12:15.280Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b08f7f6f4fb6dccbdfb1db0dc68d5340","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.287Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-24cc52a3df06bcd173e8eec5: done: 0.018134 sec","job_status":"done","queue_duration_s":2.3158,"scheduling_latency_s":2.315823,"redis_calls":10,"redis_duration_s":0.002369,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00053,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001396,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341384,"duration_s":0.018134,"completed_at":"2026-06-30T07:12:17.621Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000635,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446vm-test-run-nixbot-gitlab> gitlab # [ 161.020055] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"57becd55c006c73ef0d74737","created_at":"2026-06-30T07:12:15.296Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9882c7f9683067b3e13a4ac0fcb94387","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-30T07:12:15.305Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-57becd55c006c73ef0d74737: start","job_status":"start","queue_duration_s":2.317559,"scheduling_latency_s":2.31758}1447vm-test-run-nixbot-gitlab> gitlab # [ 161.021296] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.633Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1448vm-test-run-nixbot-gitlab> gitlab # [ 161.022241] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"57becd55c006c73ef0d74737","created_at":"2026-06-30T07:12:15.296Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9882c7f9683067b3e13a4ac0fcb94387","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-30T07:12:15.305Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-57becd55c006c73ef0d74737: done: 0.016747 sec","job_status":"done","queue_duration_s":2.317559,"scheduling_latency_s":2.31758,"redis_calls":10,"redis_duration_s":0.002385,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000483,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001338,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000564,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349486,"duration_s":0.016747,"completed_at":"2026-06-30T07:12:17.639Z","load_balancing_strategy":"primary","db_duration_s":0.000417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449vm-test-run-nixbot-gitlab> gitlab # [ 161.097351] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8ee4299618eee1c2629e1fbd","created_at":"2026-06-30T07:12:15.313Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c9561678d7ce1996e4aa3eb36e44832","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.321Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8ee4299618eee1c2629e1fbd: start","job_status":"start","queue_duration_s":2.319916,"scheduling_latency_s":2.319936}1450vm-test-run-nixbot-gitlab> gitlab # [ 161.098989] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8ee4299618eee1c2629e1fbd","created_at":"2026-06-30T07:12:15.313Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c9561678d7ce1996e4aa3eb36e44832","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.321Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8ee4299618eee1c2629e1fbd: done: 0.033785 sec","job_status":"done","queue_duration_s":2.319916,"scheduling_latency_s":2.319936,"redis_calls":12,"redis_duration_s":0.005746,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000746,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002694,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002306,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00078334,"exclusive_lock_hold_duration_s":0.008781004,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.033785,"completed_at":"2026-06-30T07:12:17.675Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451vm-test-run-nixbot-gitlab> gitlab # [ 161.170393] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"2ccf22136fdea6ee70beed6b","created_at":"2026-06-30T07:12:15.327Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c428f2636fa565e955bf491ec82c8c5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.335Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-2ccf22136fdea6ee70beed6b: start","job_status":"start","queue_duration_s":2.341638,"scheduling_latency_s":2.341661}1452vm-test-run-nixbot-gitlab> gitlab # [ 161.184053] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"2ccf22136fdea6ee70beed6b","created_at":"2026-06-30T07:12:15.327Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c428f2636fa565e955bf491ec82c8c5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.335Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-2ccf22136fdea6ee70beed6b: done: 0.017997 sec","job_status":"done","queue_duration_s":2.341638,"scheduling_latency_s":2.341661,"redis_calls":12,"redis_duration_s":0.002771,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000537778,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.017997,"completed_at":"2026-06-30T07:12:17.694Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453vm-test-run-nixbot-gitlab> gitlab # [ 161.273108] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"8f959668950a9bb9b2b7b7f5","created_at":"2026-06-30T07:12:15.359Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc511036e2720197eae01ce555c080f7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.367Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-8f959668950a9bb9b2b7b7f5: start","job_status":"start","queue_duration_s":2.328238,"scheduling_latency_s":2.328259}1454vm-test-run-nixbot-gitlab> gitlab # [ 161.343432] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.716Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"8f959668950a9bb9b2b7b7f5","created_at":"2026-06-30T07:12:15.359Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc511036e2720197eae01ce555c080f7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.367Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-8f959668950a9bb9b2b7b7f5: done: 0.020384 sec","job_status":"done","queue_duration_s":2.328238,"scheduling_latency_s":2.328259,"redis_calls":12,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001356,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000450336,"exclusive_lock_hold_duration_s":0.003521677,"duration_s":0.020384,"completed_at":"2026-06-30T07:12:17.716Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455vm-test-run-nixbot-gitlab> gitlab # [ 161.345291] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.718Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"01cb0aeb83562fae2a4ee633","created_at":"2026-06-30T07:12:15.375Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a745d2d1a1a6da7cfc15d8a82dc1593","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.381Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-01cb0aeb83562fae2a4ee633: start","job_status":"start","queue_duration_s":2.337081,"scheduling_latency_s":2.337101}1456vm-test-run-nixbot-gitlab> gitlab # [ 161.366374] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.739Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"01cb0aeb83562fae2a4ee633","created_at":"2026-06-30T07:12:15.375Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a745d2d1a1a6da7cfc15d8a82dc1593","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.381Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-01cb0aeb83562fae2a4ee633: done: 0.021155 sec","job_status":"done","queue_duration_s":2.337081,"scheduling_latency_s":2.337101,"redis_calls":12,"redis_duration_s":0.002881,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000508,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000486654,"exclusive_lock_hold_duration_s":0.0045712520000000005,"duration_s":0.021155,"completed_at":"2026-06-30T07:12:17.739Z","load_balancing_strategy":"primary","db_duration_s":0.0022,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457vm-test-run-nixbot-gitlab> gitlab # [ 161.468853] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.740Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fbcfb4e9bc674534880a9f83","created_at":"2026-06-30T07:12:15.386Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2caaff37634185774389923672df63ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-30T07:12:15.393Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fbcfb4e9bc674534880a9f83: start","job_status":"start","queue_duration_s":2.347005,"scheduling_latency_s":2.347026}1458vm-test-run-nixbot-gitlab> gitlab # [ 161.470266] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.766Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fbcfb4e9bc674534880a9f83","created_at":"2026-06-30T07:12:15.386Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2caaff37634185774389923672df63ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-30T07:12:15.393Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fbcfb4e9bc674534880a9f83: done: 0.025622 sec","job_status":"done","queue_duration_s":2.347005,"scheduling_latency_s":2.347026,"redis_calls":10,"redis_duration_s":0.002392,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000447,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001484,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000461,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386083,"duration_s":0.025622,"completed_at":"2026-06-30T07:12:17.766Z","load_balancing_strategy":"primary","db_duration_s":0.00279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459vm-test-run-nixbot-gitlab> gitlab # [ 161.560048] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.768Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0b03f1a1e51268bf7ec1b086","created_at":"2026-06-30T07:12:15.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-c90a40a6cdca4039","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.880Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0b03f1a1e51268bf7ec1b086: start","job_status":"start","queue_duration_s":1.88763,"scheduling_latency_s":1.887651}1460vm-test-run-nixbot-gitlab> gitlab # [ 161.561296] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.782Z","message":"No existing merge request to be cleaned up."}1461vm-test-run-nixbot-gitlab> gitlab # [ 161.562224] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.791Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0b03f1a1e51268bf7ec1b086","created_at":"2026-06-30T07:12:15.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-c90a40a6cdca4039","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.880Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0b03f1a1e51268bf7ec1b086: done: 0.023935 sec","job_status":"done","queue_duration_s":1.88763,"scheduling_latency_s":1.887651,"redis_calls":13,"redis_duration_s":0.003294,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001143,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001708,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355352,"duration_s":0.023935,"completed_at":"2026-06-30T07:12:17.791Z","load_balancing_strategy":"primary","db_duration_s":0.002857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462vm-test-run-nixbot-gitlab> gitlab # [ 161.647839] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.793Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"12bbbeef31651084fd5375b4","created_at":"2026-06-30T07:12:15.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-c90a40a6cdca4039","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.880Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-12bbbeef31651084fd5375b4: start","job_status":"start","queue_duration_s":1.913021,"scheduling_latency_s":1.913042}1463vm-test-run-nixbot-gitlab> gitlab # [ 161.649163] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.803Z","message":"No existing merge request to be cleaned up."}1464vm-test-run-nixbot-gitlab> gitlab # [ 161.650162] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.809Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"12bbbeef31651084fd5375b4","created_at":"2026-06-30T07:12:15.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-c90a40a6cdca4039","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.880Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-12bbbeef31651084fd5375b4: done: 0.016019 sec","job_status":"done","queue_duration_s":1.913021,"scheduling_latency_s":1.913042,"redis_calls":10,"redis_duration_s":0.002431,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016019,"completed_at":"2026-06-30T07:12:17.809Z","load_balancing_strategy":"primary","db_duration_s":0.001562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465vm-test-run-nixbot-gitlab> gitlab # [ 161.685758] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.810Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4c7d3e8007b47845a1f2811b","created_at":"2026-06-30T07:12:15.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-c90a40a6cdca4039","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.880Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4c7d3e8007b47845a1f2811b: start","job_status":"start","queue_duration_s":1.930496,"scheduling_latency_s":1.930516}1466vm-test-run-nixbot-gitlab> gitlab # [ 161.705438] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.820Z","message":"No existing merge request to be cleaned up."}1467vm-test-run-nixbot-gitlab> gitlab # [ 161.708098] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.826Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4c7d3e8007b47845a1f2811b","created_at":"2026-06-30T07:12:15.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-c90a40a6cdca4039","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.880Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4c7d3e8007b47845a1f2811b: done: 0.015917 sec","job_status":"done","queue_duration_s":1.930496,"scheduling_latency_s":1.930516,"redis_calls":10,"redis_duration_s":0.002323,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001377,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015917,"completed_at":"2026-06-30T07:12:17.826Z","load_balancing_strategy":"primary","db_duration_s":0.001541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468vm-test-run-nixbot-gitlab> gitlab # [ 161.793212] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.828Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c0e805c0f4e4f299d22dab73","created_at":"2026-06-30T07:12:15.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-c90a40a6cdca4039","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.880Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c0e805c0f4e4f299d22dab73: start","job_status":"start","queue_duration_s":1.948006,"scheduling_latency_s":1.94803}1469vm-test-run-nixbot-gitlab> gitlab # [ 161.794733] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.838Z","message":"No existing merge request to be cleaned up."}1470vm-test-run-nixbot-gitlab> gitlab # [ 161.795928] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.845Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c0e805c0f4e4f299d22dab73","created_at":"2026-06-30T07:12:15.858Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-c90a40a6cdca4039","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a278d9268fc8909c689db04ea6ded218","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:15.880Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c0e805c0f4e4f299d22dab73: done: 0.017589 sec","job_status":"done","queue_duration_s":1.948006,"scheduling_latency_s":1.94803,"redis_calls":10,"redis_duration_s":0.003941,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00151,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002431,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017589,"completed_at":"2026-06-30T07:12:17.845Z","load_balancing_strategy":"primary","db_duration_s":0.001676,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471vm-test-run-nixbot-gitlab> gitlab # [ 161.872355] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e52bf367763c7d20c99b434e","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e52bf367763c7d20c99b434e: start","job_status":"start","queue_duration_s":0.253951,"scheduling_latency_s":0.253973}1472vm-test-run-nixbot-gitlab> gitlab # [ 161.873457] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e52bf367763c7d20c99b434e","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e52bf367763c7d20c99b434e: done: 0.021516 sec","job_status":"done","queue_duration_s":0.253951,"scheduling_latency_s":0.253973,"redis_calls":16,"redis_duration_s":0.003941,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001143,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002302,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000366527,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021516,"completed_at":"2026-06-30T07:12:17.868Z","load_balancing_strategy":"primary","db_duration_s":0.000412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473vm-test-run-nixbot-gitlab> gitlab # [ 161.936268] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d82d5a708026decbc97fd278","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d82d5a708026decbc97fd278: start","job_status":"start","queue_duration_s":0.276916,"scheduling_latency_s":0.276936}1474vm-test-run-nixbot-gitlab> gitlab # [ 161.955564] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d82d5a708026decbc97fd278","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d82d5a708026decbc97fd278: done: 0.015747 sec","job_status":"done","queue_duration_s":0.276916,"scheduling_latency_s":0.276936,"redis_calls":13,"redis_duration_s":0.003065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015747,"completed_at":"2026-06-30T07:12:17.886Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475vm-test-run-nixbot-gitlab> gitlab # [ 162.019100] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f414474e6fa52f6ed2649df","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f414474e6fa52f6ed2649df: start","job_status":"start","queue_duration_s":0.293977,"scheduling_latency_s":0.293999}1476vm-test-run-nixbot-gitlab> gitlab # [ 162.038805] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f414474e6fa52f6ed2649df","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f414474e6fa52f6ed2649df: done: 0.015587 sec","job_status":"done","queue_duration_s":0.293977,"scheduling_latency_s":0.293999,"redis_calls":13,"redis_duration_s":0.003111,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015587,"completed_at":"2026-06-30T07:12:17.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477vm-test-run-nixbot-gitlab> gitlab # [ 162.102311] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bec6e59f2386bb01e00125f","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bec6e59f2386bb01e00125f: start","job_status":"start","queue_duration_s":0.311002,"scheduling_latency_s":0.311023}1478vm-test-run-nixbot-gitlab> gitlab # [ 162.122253] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bec6e59f2386bb01e00125f","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bec6e59f2386bb01e00125f: done: 0.015756 sec","job_status":"done","queue_duration_s":0.311002,"scheduling_latency_s":0.311023,"redis_calls":13,"redis_duration_s":0.0032099999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002289,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015756,"completed_at":"2026-06-30T07:12:17.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479vm-test-run-nixbot-gitlab> gitlab # [ 162.185797] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"942368c027266140c46c4f55","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-942368c027266140c46c4f55: start","job_status":"start","queue_duration_s":0.328078,"scheduling_latency_s":0.3281}1480vm-test-run-nixbot-gitlab> gitlab # [ 162.205930] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"942368c027266140c46c4f55","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-942368c027266140c46c4f55: done: 0.015724 sec","job_status":"done","queue_duration_s":0.328078,"scheduling_latency_s":0.3281,"redis_calls":13,"redis_duration_s":0.003249,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015724,"completed_at":"2026-06-30T07:12:17.937Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481vm-test-run-nixbot-gitlab> gitlab # [ 162.268971] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"811d5e4752e57f069b5d7b56","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-811d5e4752e57f069b5d7b56: start","job_status":"start","queue_duration_s":0.345129,"scheduling_latency_s":0.345152}1482vm-test-run-nixbot-gitlab> gitlab # [ 162.288943] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"811d5e4752e57f069b5d7b56","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-811d5e4752e57f069b5d7b56: done: 0.015615 sec","job_status":"done","queue_duration_s":0.345129,"scheduling_latency_s":0.345152,"redis_calls":13,"redis_duration_s":0.002935,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015615,"completed_at":"2026-06-30T07:12:17.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483vm-test-run-nixbot-gitlab> gitlab # [ 162.352139] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68b5dcb223783f18b6c58353","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68b5dcb223783f18b6c58353: start","job_status":"start","queue_duration_s":0.362015,"scheduling_latency_s":0.362037}1484vm-test-run-nixbot-gitlab> gitlab # [ 162.372116] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68b5dcb223783f18b6c58353","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68b5dcb223783f18b6c58353: done: 0.015475 sec","job_status":"done","queue_duration_s":0.362015,"scheduling_latency_s":0.362037,"redis_calls":13,"redis_duration_s":0.003027,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015475,"completed_at":"2026-06-30T07:12:17.970Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485vm-test-run-nixbot-gitlab> gitlab # [ 162.435305] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c642e3ea73a40aa2175c8710","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c642e3ea73a40aa2175c8710: start","job_status":"start","queue_duration_s":0.378968,"scheduling_latency_s":0.378988}1486vm-test-run-nixbot-gitlab> gitlab # [ 162.521340] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c642e3ea73a40aa2175c8710","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c642e3ea73a40aa2175c8710: done: 0.020325 sec","job_status":"done","queue_duration_s":0.378968,"scheduling_latency_s":0.378988,"redis_calls":13,"redis_duration_s":0.003503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00245,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020325,"completed_at":"2026-06-30T07:12:17.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487vm-test-run-nixbot-gitlab> gitlab # [ 162.523058] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:17.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67004c0dc8bd4e0926365a9c","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67004c0dc8bd4e0926365a9c: start","job_status":"start","queue_duration_s":0.401263,"scheduling_latency_s":0.401297}1488vm-test-run-nixbot-gitlab> gitlab # [ 162.535921] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67004c0dc8bd4e0926365a9c","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67004c0dc8bd4e0926365a9c: done: 0.01736 sec","job_status":"done","queue_duration_s":0.401263,"scheduling_latency_s":0.401297,"redis_calls":13,"redis_duration_s":0.0028599999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01736,"completed_at":"2026-06-30T07:12:18.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489vm-test-run-nixbot-gitlab> gitlab # [ 162.641924] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d73481c5e05914ab5a5c490","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d73481c5e05914ab5a5c490: start","job_status":"start","queue_duration_s":0.420115,"scheduling_latency_s":0.420139}1490vm-test-run-nixbot-gitlab> gitlab # [ 162.643255] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d73481c5e05914ab5a5c490","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d73481c5e05914ab5a5c490: done: 0.01852 sec","job_status":"done","queue_duration_s":0.420115,"scheduling_latency_s":0.420139,"redis_calls":13,"redis_duration_s":0.003648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002607,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01852,"completed_at":"2026-06-30T07:12:18.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491vm-test-run-nixbot-gitlab> gitlab # [ 162.679764] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f4d04d8c0651cdbd6396124","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f4d04d8c0651cdbd6396124: start","job_status":"start","queue_duration_s":0.440378,"scheduling_latency_s":0.440406}1492vm-test-run-nixbot-gitlab> gitlab # [ 162.699825] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f4d04d8c0651cdbd6396124","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f4d04d8c0651cdbd6396124: done: 0.017484 sec","job_status":"done","queue_duration_s":0.440378,"scheduling_latency_s":0.440406,"redis_calls":13,"redis_duration_s":0.00307,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017484,"completed_at":"2026-06-30T07:12:18.051Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493vm-test-run-nixbot-gitlab> gitlab # [ 162.762995] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"384c3d71e06686d0ba96a52c","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-384c3d71e06686d0ba96a52c: start","job_status":"start","queue_duration_s":0.459473,"scheduling_latency_s":0.459495}1494vm-test-run-nixbot-gitlab> gitlab # [ 162.783104] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"384c3d71e06686d0ba96a52c","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-384c3d71e06686d0ba96a52c: done: 0.016459 sec","job_status":"done","queue_duration_s":0.459473,"scheduling_latency_s":0.459495,"redis_calls":13,"redis_duration_s":0.0028380000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016459,"completed_at":"2026-06-30T07:12:18.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495vm-test-run-nixbot-gitlab> gitlab # [ 162.856153] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9e5746a1d2441972d9240a2","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9e5746a1d2441972d9240a2: start","job_status":"start","queue_duration_s":0.477417,"scheduling_latency_s":0.47744}1496vm-test-run-nixbot-gitlab> gitlab # [ 162.864113] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9e5746a1d2441972d9240a2","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9e5746a1d2441972d9240a2: done: 0.016307 sec","job_status":"done","queue_duration_s":0.477417,"scheduling_latency_s":0.47744,"redis_calls":13,"redis_duration_s":0.002863,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016307,"completed_at":"2026-06-30T07:12:18.087Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497vm-test-run-nixbot-gitlab> gitlab # [ 162.947618] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"140e842d8b1b12ef5cf960a0","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-140e842d8b1b12ef5cf960a0: start","job_status":"start","queue_duration_s":0.495066,"scheduling_latency_s":0.495089}1498vm-test-run-nixbot-gitlab> gitlab # [ 162.948818] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"140e842d8b1b12ef5cf960a0","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-140e842d8b1b12ef5cf960a0: done: 0.016402 sec","job_status":"done","queue_duration_s":0.495066,"scheduling_latency_s":0.495089,"redis_calls":13,"redis_duration_s":0.002922,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016402,"completed_at":"2026-06-30T07:12:18.104Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499vm-test-run-nixbot-gitlab> gitlab # [ 163.008231] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3be41f353cf71c8bd7dc09ef","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3be41f353cf71c8bd7dc09ef: start","job_status":"start","queue_duration_s":0.512854,"scheduling_latency_s":0.512877}1500vm-test-run-nixbot-gitlab> gitlab # [ 163.028341] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3be41f353cf71c8bd7dc09ef","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3be41f353cf71c8bd7dc09ef: done: 0.019001 sec","job_status":"done","queue_duration_s":0.512854,"scheduling_latency_s":0.512877,"redis_calls":13,"redis_duration_s":0.003123,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019001,"completed_at":"2026-06-30T07:12:18.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501vm-test-run-nixbot-gitlab> gitlab # [ 163.099241] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"748fccbf0e45a7a51493db9d","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-748fccbf0e45a7a51493db9d: start","job_status":"start","queue_duration_s":0.533386,"scheduling_latency_s":0.533409}1502vm-test-run-nixbot-gitlab> gitlab # [ 163.108449] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"748fccbf0e45a7a51493db9d","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-748fccbf0e45a7a51493db9d: done: 0.015333 sec","job_status":"done","queue_duration_s":0.533386,"scheduling_latency_s":0.533409,"redis_calls":13,"redis_duration_s":0.0029679999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015333,"completed_at":"2026-06-30T07:12:18.142Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503vm-test-run-nixbot-gitlab> gitlab # [ 163.191244] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e477d8ba88764e65c21d2563","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e477d8ba88764e65c21d2563: start","job_status":"start","queue_duration_s":0.549965,"scheduling_latency_s":0.549985}1504vm-test-run-nixbot-gitlab> gitlab # [ 163.192589] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e477d8ba88764e65c21d2563","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e477d8ba88764e65c21d2563: done: 0.015699 sec","job_status":"done","queue_duration_s":0.549965,"scheduling_latency_s":0.549985,"redis_calls":13,"redis_duration_s":0.00291,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015699,"completed_at":"2026-06-30T07:12:18.159Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505vm-test-run-nixbot-gitlab> gitlab # [ 163.263065] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8df3b099453fb1ef89984b4b","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8df3b099453fb1ef89984b4b: start","job_status":"start","queue_duration_s":0.567019,"scheduling_latency_s":0.56704}1506vm-test-run-nixbot-gitlab> gitlab # [ 163.271275] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8df3b099453fb1ef89984b4b","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8df3b099453fb1ef89984b4b: done: 0.015474 sec","job_status":"done","queue_duration_s":0.567019,"scheduling_latency_s":0.56704,"redis_calls":13,"redis_duration_s":0.0029809999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015474,"completed_at":"2026-06-30T07:12:18.175Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507vm-test-run-nixbot-gitlab> gitlab # [ 163.386297] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.177Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a613466fa3f9f3ec3fdcc901","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a613466fa3f9f3ec3fdcc901: start","job_status":"start","queue_duration_s":0.584084,"scheduling_latency_s":0.584106}1508vm-test-run-nixbot-gitlab> gitlab # [ 163.388053] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a613466fa3f9f3ec3fdcc901","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a613466fa3f9f3ec3fdcc901: done: 0.015686 sec","job_status":"done","queue_duration_s":0.584084,"scheduling_latency_s":0.584106,"redis_calls":13,"redis_duration_s":0.0031279999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002237,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015686,"completed_at":"2026-06-30T07:12:18.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509vm-test-run-nixbot-gitlab> gitlab # [ 163.415392] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f316a566d6c3b53191f9c1a","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f316a566d6c3b53191f9c1a: start","job_status":"start","queue_duration_s":0.60126,"scheduling_latency_s":0.601282}1510vm-test-run-nixbot-gitlab> gitlab # [ 163.435429] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f316a566d6c3b53191f9c1a","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f316a566d6c3b53191f9c1a: done: 0.015628 sec","job_status":"done","queue_duration_s":0.60126,"scheduling_latency_s":0.601282,"redis_calls":13,"redis_duration_s":0.003077,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015628,"completed_at":"2026-06-30T07:12:18.210Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511vm-test-run-nixbot-gitlab> gitlab # [ 163.531209] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0480da30dd9f17b0d37dc98","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0480da30dd9f17b0d37dc98: start","job_status":"start","queue_duration_s":0.61859,"scheduling_latency_s":0.618613}1512vm-test-run-nixbot-gitlab> gitlab # [ 163.532815] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0480da30dd9f17b0d37dc98","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0480da30dd9f17b0d37dc98: done: 0.015679 sec","job_status":"done","queue_duration_s":0.61859,"scheduling_latency_s":0.618613,"redis_calls":13,"redis_duration_s":0.002911,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015679,"completed_at":"2026-06-30T07:12:18.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513vm-test-run-nixbot-gitlab> gitlab # [ 163.615368] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"543d106d0ca9a7f6a441ecfd","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-543d106d0ca9a7f6a441ecfd: start","job_status":"start","queue_duration_s":0.635906,"scheduling_latency_s":0.635927}1514vm-test-run-nixbot-gitlab> gitlab # [ 163.616996] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"543d106d0ca9a7f6a441ecfd","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-543d106d0ca9a7f6a441ecfd: done: 0.015563 sec","job_status":"done","queue_duration_s":0.635906,"scheduling_latency_s":0.635927,"redis_calls":13,"redis_duration_s":0.002955,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015563,"completed_at":"2026-06-30T07:12:18.244Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515vm-test-run-nixbot-gitlab> gitlab # [ 163.695326] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd2140c1da41778b890b5450","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd2140c1da41778b890b5450: start","job_status":"start","queue_duration_s":0.652965,"scheduling_latency_s":0.652986}1516vm-test-run-nixbot-gitlab> gitlab # [ 163.697053] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd2140c1da41778b890b5450","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd2140c1da41778b890b5450: done: 0.015649 sec","job_status":"done","queue_duration_s":0.652965,"scheduling_latency_s":0.652986,"redis_calls":13,"redis_duration_s":0.00298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015649,"completed_at":"2026-06-30T07:12:18.262Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517vm-test-run-nixbot-gitlab> gitlab # [ 163.740391] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9eba7adf16d888a8659c002b","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9eba7adf16d888a8659c002b: start","job_status":"start","queue_duration_s":0.669924,"scheduling_latency_s":0.669947}1518vm-test-run-nixbot-gitlab> gitlab # [ 163.760504] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9eba7adf16d888a8659c002b","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9eba7adf16d888a8659c002b: done: 0.017396 sec","job_status":"done","queue_duration_s":0.669924,"scheduling_latency_s":0.669947,"redis_calls":13,"redis_duration_s":0.003373,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00116,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017396,"completed_at":"2026-06-30T07:12:18.280Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519vm-test-run-nixbot-gitlab> gitlab # [ 163.882971] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bc5101d0f3221e00644c2c4","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bc5101d0f3221e00644c2c4: start","job_status":"start","queue_duration_s":0.688648,"scheduling_latency_s":0.688671}1520vm-test-run-nixbot-gitlab> gitlab # [ 163.884306] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bc5101d0f3221e00644c2c4","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bc5101d0f3221e00644c2c4: done: 0.01585 sec","job_status":"done","queue_duration_s":0.688648,"scheduling_latency_s":0.688671,"redis_calls":13,"redis_duration_s":0.003056,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01585,"completed_at":"2026-06-30T07:12:18.297Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521vm-test-run-nixbot-gitlab> gitlab # [ 163.961343] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e040d24253ccd57a81833435","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e040d24253ccd57a81833435: start","job_status":"start","queue_duration_s":0.705912,"scheduling_latency_s":0.705934}1522vm-test-run-nixbot-gitlab> gitlab # [ 163.962921] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e040d24253ccd57a81833435","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e040d24253ccd57a81833435: done: 0.020445 sec","job_status":"done","queue_duration_s":0.705912,"scheduling_latency_s":0.705934,"redis_calls":13,"redis_duration_s":0.003454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001003,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002451,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020445,"completed_at":"2026-06-30T07:12:18.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523vm-test-run-nixbot-gitlab> gitlab # [ 163.984784] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87c5a8cd49045113f8c6a1c8","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87c5a8cd49045113f8c6a1c8: start","job_status":"start","queue_duration_s":0.728038,"scheduling_latency_s":0.728061}1524vm-test-run-nixbot-gitlab> gitlab # [ 164.004840] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87c5a8cd49045113f8c6a1c8","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87c5a8cd49045113f8c6a1c8: done: 0.01707 sec","job_status":"done","queue_duration_s":0.728038,"scheduling_latency_s":0.728061,"redis_calls":13,"redis_duration_s":0.003247,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00226,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01707,"completed_at":"2026-06-30T07:12:18.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525vm-test-run-nixbot-gitlab> gitlab # [ 164.092301] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfe128e6f8537e0a5eceafe6","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfe128e6f8537e0a5eceafe6: start","job_status":"start","queue_duration_s":0.746483,"scheduling_latency_s":0.746506}1526vm-test-run-nixbot-gitlab> gitlab # [ 164.093884] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfe128e6f8537e0a5eceafe6","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfe128e6f8537e0a5eceafe6: done: 0.016622 sec","job_status":"done","queue_duration_s":0.746483,"scheduling_latency_s":0.746506,"redis_calls":13,"redis_duration_s":0.002912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016622,"completed_at":"2026-06-30T07:12:18.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527vm-test-run-nixbot-gitlab> gitlab # [ 164.202279] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b480a6fa051231ccbf855866","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b480a6fa051231ccbf855866: start","job_status":"start","queue_duration_s":0.764485,"scheduling_latency_s":0.764507}1528vm-test-run-nixbot-gitlab> gitlab # [ 164.203870] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b480a6fa051231ccbf855866","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b480a6fa051231ccbf855866: done: 0.016143 sec","job_status":"done","queue_duration_s":0.764485,"scheduling_latency_s":0.764507,"redis_calls":13,"redis_duration_s":0.00304,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016143,"completed_at":"2026-06-30T07:12:18.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529vm-test-run-nixbot-gitlab> gitlab # [ 164.288393] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea2b897fa5c676f84b7698fe","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea2b897fa5c676f84b7698fe: start","job_status":"start","queue_duration_s":0.782126,"scheduling_latency_s":0.782148}1530vm-test-run-nixbot-gitlab> gitlab # [ 164.290061] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea2b897fa5c676f84b7698fe","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea2b897fa5c676f84b7698fe: done: 0.01593 sec","job_status":"done","queue_duration_s":0.782126,"scheduling_latency_s":0.782148,"redis_calls":13,"redis_duration_s":0.00287,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01593,"completed_at":"2026-06-30T07:12:18.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531vm-test-run-nixbot-gitlab> gitlab # [ 164.312265] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:18.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"275c374197d00c1e4237356e","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-275c374197d00c1e4237356e: start","job_status":"start","queue_duration_s":0.799572,"scheduling_latency_s":0.799594}1532vm-test-run-nixbot-gitlab> gitlab # [ 164.661959] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 4138418189 wd_nsec: 41384157221533vm-test-run-nixbot-gitlab> gitlab # [ 164.414431] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"275c374197d00c1e4237356e","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-275c374197d00c1e4237356e: done: 2.953511 sec","job_status":"done","queue_duration_s":0.799572,"scheduling_latency_s":0.799594,"redis_calls":13,"redis_duration_s":0.004388,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001779,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002609,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":2.953511,"completed_at":"2026-06-30T07:12:21.346Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534vm-test-run-nixbot-gitlab> gitlab # [ 164.416233] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b3d3063bae6be6f9899c55a","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b3d3063bae6be6f9899c55a: start","job_status":"start","queue_duration_s":3.75475,"scheduling_latency_s":3.754781}1535vm-test-run-nixbot-gitlab> gitlab # [ 164.492500] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b3d3063bae6be6f9899c55a","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b3d3063bae6be6f9899c55a: done: 0.015911 sec","job_status":"done","queue_duration_s":3.75475,"scheduling_latency_s":3.754781,"redis_calls":13,"redis_duration_s":0.003033,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015911,"completed_at":"2026-06-30T07:12:21.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536vm-test-run-nixbot-gitlab> gitlab # [ 164.494953] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b22f3fbd4e8f4cfd794cc1ec","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b22f3fbd4e8f4cfd794cc1ec: start","job_status":"start","queue_duration_s":3.772381,"scheduling_latency_s":3.772403}1537vm-test-run-nixbot-gitlab> gitlab # [ 164.567670] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b22f3fbd4e8f4cfd794cc1ec","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b22f3fbd4e8f4cfd794cc1ec: done: 0.015326 sec","job_status":"done","queue_duration_s":3.772381,"scheduling_latency_s":3.772403,"redis_calls":13,"redis_duration_s":0.002857,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015326,"completed_at":"2026-06-30T07:12:21.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538vm-test-run-nixbot-gitlab> gitlab # [ 164.569370] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77c03b5b64f49a0fd579f4ff","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77c03b5b64f49a0fd579f4ff: start","job_status":"start","queue_duration_s":3.788983,"scheduling_latency_s":3.789005}1539vm-test-run-nixbot-gitlab> gitlab # [ 164.646238] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77c03b5b64f49a0fd579f4ff","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77c03b5b64f49a0fd579f4ff: done: 0.015192 sec","job_status":"done","queue_duration_s":3.788983,"scheduling_latency_s":3.789005,"redis_calls":13,"redis_duration_s":0.002895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015192,"completed_at":"2026-06-30T07:12:21.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540vm-test-run-nixbot-gitlab> gitlab # [ 164.715375] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4167860b8fe3cd74586e9f98","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4167860b8fe3cd74586e9f98: start","job_status":"start","queue_duration_s":3.805538,"scheduling_latency_s":3.805561}1541vm-test-run-nixbot-gitlab> gitlab # [ 164.716470] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4167860b8fe3cd74586e9f98","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4167860b8fe3cd74586e9f98: done: 0.01524 sec","job_status":"done","queue_duration_s":3.805538,"scheduling_latency_s":3.805561,"redis_calls":13,"redis_duration_s":0.002988,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01524,"completed_at":"2026-06-30T07:12:21.414Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542vm-test-run-nixbot-gitlab> gitlab # [ 164.796278] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"831b9dafaac93d88d297c943","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-831b9dafaac93d88d297c943: start","job_status":"start","queue_duration_s":3.822489,"scheduling_latency_s":3.82251}1543vm-test-run-nixbot-gitlab> gitlab # [ 164.797943] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"831b9dafaac93d88d297c943","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-831b9dafaac93d88d297c943: done: 0.0152 sec","job_status":"done","queue_duration_s":3.822489,"scheduling_latency_s":3.82251,"redis_calls":13,"redis_duration_s":0.0029419999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0152,"completed_at":"2026-06-30T07:12:21.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544vm-test-run-nixbot-gitlab> gitlab # [ 164.812739] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b3c8731e5319424019d29aa","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b3c8731e5319424019d29aa: start","job_status":"start","queue_duration_s":3.838959,"scheduling_latency_s":3.838981}1545vm-test-run-nixbot-gitlab> gitlab # [ 164.832911] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b3c8731e5319424019d29aa","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b3c8731e5319424019d29aa: done: 0.015325 sec","job_status":"done","queue_duration_s":3.838959,"scheduling_latency_s":3.838981,"redis_calls":13,"redis_duration_s":0.002933,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015325,"completed_at":"2026-06-30T07:12:21.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546vm-test-run-nixbot-gitlab> gitlab # [ 164.920075] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f9d946983cc514b1d7eb568","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f9d946983cc514b1d7eb568: start","job_status":"start","queue_duration_s":3.855576,"scheduling_latency_s":3.855597}1547vm-test-run-nixbot-gitlab> gitlab # [ 164.921382] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f9d946983cc514b1d7eb568","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f9d946983cc514b1d7eb568: done: 0.015092 sec","job_status":"done","queue_duration_s":3.855576,"scheduling_latency_s":3.855597,"redis_calls":13,"redis_duration_s":0.002934,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015092,"completed_at":"2026-06-30T07:12:21.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548vm-test-run-nixbot-gitlab> gitlab # [ 164.976682] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b256a315313d22e85d89bd0","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b256a315313d22e85d89bd0: start","job_status":"start","queue_duration_s":3.871967,"scheduling_latency_s":3.871989}1549vm-test-run-nixbot-gitlab> gitlab # [ 165.059092] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b256a315313d22e85d89bd0","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b256a315313d22e85d89bd0: done: 0.015307 sec","job_status":"done","queue_duration_s":3.871967,"scheduling_latency_s":3.871989,"redis_calls":13,"redis_duration_s":0.0029029999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015307,"completed_at":"2026-06-30T07:12:21.480Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550vm-test-run-nixbot-gitlab> gitlab # [ 165.061211] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8f6bca0db4fb8cceae4dbe2","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8f6bca0db4fb8cceae4dbe2: start","job_status":"start","queue_duration_s":3.888998,"scheduling_latency_s":3.88902}1551vm-test-run-nixbot-gitlab> gitlab # [ 165.077289] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.498Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8f6bca0db4fb8cceae4dbe2","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8f6bca0db4fb8cceae4dbe2: done: 0.015547 sec","job_status":"done","queue_duration_s":3.888998,"scheduling_latency_s":3.88902,"redis_calls":13,"redis_duration_s":0.002933,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015547,"completed_at":"2026-06-30T07:12:21.498Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552vm-test-run-nixbot-gitlab> gitlab # [ 165.145361] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0053e64cf0f4153262991ee1","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0053e64cf0f4153262991ee1: start","job_status":"start","queue_duration_s":3.905867,"scheduling_latency_s":3.905888}1553vm-test-run-nixbot-gitlab> gitlab # [ 165.157915] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0053e64cf0f4153262991ee1","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0053e64cf0f4153262991ee1: done: 0.015474 sec","job_status":"done","queue_duration_s":3.905867,"scheduling_latency_s":3.905888,"redis_calls":13,"redis_duration_s":0.0034590000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002514,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015474,"completed_at":"2026-06-30T07:12:21.514Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554vm-test-run-nixbot-gitlab> gitlab # [ 165.234061] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3701961150630efbb5f7efbf","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3701961150630efbb5f7efbf: start","job_status":"start","queue_duration_s":3.922706,"scheduling_latency_s":3.922728}1555vm-test-run-nixbot-gitlab> gitlab # [ 165.238486] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3701961150630efbb5f7efbf","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3701961150630efbb5f7efbf: done: 0.016072 sec","job_status":"done","queue_duration_s":3.922706,"scheduling_latency_s":3.922728,"redis_calls":13,"redis_duration_s":0.003201,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002244,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016072,"completed_at":"2026-06-30T07:12:21.532Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556vm-test-run-nixbot-gitlab> gitlab # [ 165.336148] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.533Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c0f81bdfd421533f5054b51","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c0f81bdfd421533f5054b51: start","job_status":"start","queue_duration_s":3.940058,"scheduling_latency_s":3.94008}1557vm-test-run-nixbot-gitlab> gitlab # [ 165.337443] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c0f81bdfd421533f5054b51","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c0f81bdfd421533f5054b51: done: 0.015701 sec","job_status":"done","queue_duration_s":3.940058,"scheduling_latency_s":3.94008,"redis_calls":13,"redis_duration_s":0.003005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015701,"completed_at":"2026-06-30T07:12:21.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558vm-test-run-nixbot-gitlab> gitlab # [ 165.382231] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7840b8486ac03111fc169dbf","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7840b8486ac03111fc169dbf: start","job_status":"start","queue_duration_s":3.957782,"scheduling_latency_s":3.957804}1559vm-test-run-nixbot-gitlab> gitlab # [ 165.456113] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7840b8486ac03111fc169dbf","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7840b8486ac03111fc169dbf: done: 0.015688 sec","job_status":"done","queue_duration_s":3.957782,"scheduling_latency_s":3.957804,"redis_calls":13,"redis_duration_s":0.00349,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015688,"completed_at":"2026-06-30T07:12:21.566Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560vm-test-run-nixbot-gitlab> gitlab # [ 165.462901] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea9c92a6501da6548e374ba8","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea9c92a6501da6548e374ba8: start","job_status":"start","queue_duration_s":3.974775,"scheduling_latency_s":3.974797}1561vm-test-run-nixbot-gitlab> gitlab # [ 165.533982] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea9c92a6501da6548e374ba8","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea9c92a6501da6548e374ba8: done: 0.015401 sec","job_status":"done","queue_duration_s":3.974775,"scheduling_latency_s":3.974797,"redis_calls":13,"redis_duration_s":0.002941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015401,"completed_at":"2026-06-30T07:12:21.583Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562vm-test-run-nixbot-gitlab> gitlab # [ 165.543211] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a09b1e31726a47852c3c8bbc","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a09b1e31726a47852c3c8bbc: start","job_status":"start","queue_duration_s":3.991495,"scheduling_latency_s":3.991515}1563vm-test-run-nixbot-gitlab> gitlab # [ 165.563302] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a09b1e31726a47852c3c8bbc","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a09b1e31726a47852c3c8bbc: done: 0.015289 sec","job_status":"done","queue_duration_s":3.991495,"scheduling_latency_s":3.991515,"redis_calls":13,"redis_duration_s":0.002888,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_[ 165.831271] sched: DL replenish lagged too much1564vm-test-run-nixbot-gitlab> gitlab # s":0.0,"cpu_s":0.010316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015289,"completed_at":"2026-06-30T07:12:21.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565vm-test-run-nixbot-gitlab> gitlab # [ 165.639320] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecca24a9cea896c470cab550","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecca24a9cea896c470cab550: start","job_status":"start","queue_duration_s":4.008044,"scheduling_latency_s":4.008065}1566vm-test-run-nixbot-gitlab> gitlab # [ 165.644170] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecca24a9cea896c470cab550","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecca24a9cea896c470cab550: done: 0.056707 sec","job_status":"done","queue_duration_s":4.008044,"scheduling_latency_s":4.008065,"redis_calls":13,"redis_duration_s":0.003022,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.056707,"completed_at":"2026-06-30T07:12:21.658Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567vm-test-run-nixbot-gitlab> gitlab # [ 165.726286] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9008754db5908248b387c5b5","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9008754db5908248b387c5b5: start","job_status":"start","queue_duration_s":4.066643,"scheduling_latency_s":4.066667}1568vm-test-run-nixbot-gitlab> gitlab # [ 165.728055] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9008754db5908248b387c5b5","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9008754db5908248b387c5b5: done: 0.015488 sec","job_status":"done","queue_duration_s":4.066643,"scheduling_latency_s":4.066667,"redis_calls":13,"redis_duration_s":0.002953,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015488,"completed_at":"2026-06-30T07:12:21.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569vm-test-run-nixbot-gitlab> gitlab # [ 165.797295] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47e91a492d39d03017907084","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47e91a492d39d03017907084: start","job_status":"start","queue_duration_s":4.083438,"scheduling_latency_s":4.08346}1570vm-test-run-nixbot-gitlab> gitlab # [ 165.806307] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47e91a492d39d03017907084","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47e91a492d39d03017907084: done: 0.015272 sec","job_status":"done","queue_duration_s":4.083438,"scheduling_latency_s":4.08346,"redis_calls":13,"redis_duration_s":0.002852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015272,"completed_at":"2026-06-30T07:12:21.692Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571vm-test-run-nixbot-gitlab> gitlab # [ 165.924955] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe4308195a4be87a5c409811","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe4308195a4be87a5c409811: start","job_status":"start","queue_duration_s":4.09998,"scheduling_latency_s":4.100001}1572vm-test-run-nixbot-gitlab> gitlab # [ 165.926300] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:12:21.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe4308195a4be87a5c409811","created_at":"2026-06-30T07:12:17.526Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-ea2ad73ab4884c75","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4b0d0dd0c46fdb774233981504d0cc40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:12:17.593Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe4308195a4be87a5c409811: done: 0.015148 sec","job_status":"done","queue_duration_s":4.09998,"scheduling_latency_s":4.100001,"redis_calls":13,"redis_duration_s":0.0029249999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015148,"completed_at":"2026-06-30T07:12:21.708Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573vm-test-run-nixbot-gitlab> gitlab # [ 184.582696] setup-gitlab-start[2769]: W, [2026-06-30T07:12:41.514569 #2769] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1574vm-test-run-nixbot-gitlab> gitlab # [ 204.704665] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:01.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"53ba9c3b06b4a7863df87c16","created_at":"2026-06-30T07:13:01.602Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"010a66b86744a6d4ebd75b3be5b60a97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.632Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-53ba9c3b06b4a7863df87c16: start","job_status":"start","queue_duration_s":0.003219,"scheduling_latency_s":0.003293}1575vm-test-run-nixbot-gitlab> gitlab # [ 204.802516] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:01.734Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1576vm-test-run-nixbot-gitlab> gitlab # [ 204.839138] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:01.770Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"53ba9c3b06b4a7863df87c16","created_at":"2026-06-30T07:13:01.602Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"010a66b86744a6d4ebd75b3be5b60a97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.632Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-53ba9c3b06b4a7863df87c16: done: 0.135332 sec","job_status":"done","queue_duration_s":0.003219,"scheduling_latency_s":0.003293,"redis_calls":8,"redis_duration_s":0.034331,"redis_read_bytes":133,"redis_write_bytes":830,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001093,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":64,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.03151,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001728,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.135332,"completed_at":"2026-06-30T07:13:01.770Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577vm-test-run-nixbot-gitlab> gitlab # [ 204.924324] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:01.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"44603c0d4c1671110b23e151","created_at":"2026-06-30T07:13:01.690Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c88cd549008cc0789bb903aa32348bf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.697Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-44603c0d4c1671110b23e151: start","job_status":"start","queue_duration_s":0.159114,"scheduling_latency_s":0.159154}1578vm-test-run-nixbot-gitlab> gitlab # [ 205.558378] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"44603c0d4c1671110b23e151","created_at":"2026-06-30T07:13:01.690Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c88cd549008cc0789bb903aa32348bf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.697Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-44603c0d4c1671110b23e151: done: 0.634014 sec","job_status":"done","queue_duration_s":0.159114,"scheduling_latency_s":0.159154,"redis_calls":9,"redis_duration_s":0.014012,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003336,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":123,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":123,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.094,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.094,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.280542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001689042,"exclusive_lock_hold_duration_s":0.541893174,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.634014,"completed_at":"2026-06-30T07:13:02.490Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.321211,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1579vm-test-run-nixbot-gitlab> gitlab # [ 205.628426] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2799a43a46247d4d2f55b568","created_at":"2026-06-30T07:13:01.731Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.757Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2799a43a46247d4d2f55b568: start","job_status":"start","queue_duration_s":0.802698,"scheduling_latency_s":0.802794}1580vm-test-run-nixbot-gitlab> gitlab # [ 205.698310] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.630Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2799a43a46247d4d2f55b568","created_at":"2026-06-30T07:13:01.731Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.757Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2799a43a46247d4d2f55b568: done: 0.070664 sec","job_status":"done","queue_duration_s":0.802698,"scheduling_latency_s":0.802794,"redis_calls":14,"redis_duration_s":0.007919,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000781,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005574,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001564,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.070664,"completed_at":"2026-06-30T07:13:02.630Z","load_balancing_strategy":"primary","db_duration_s":0.025471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1581vm-test-run-nixbot-gitlab> gitlab # [ 205.703758] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.635Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5f6b25f1edafa7f8a6ad3359","created_at":"2026-06-30T07:13:01.850Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d71ff86ab464da169978e8a4cb3114e","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.855Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5f6b25f1edafa7f8a6ad3359: start","job_status":"start","queue_duration_s":0.780074,"scheduling_latency_s":0.780208}1582vm-test-run-nixbot-gitlab> gitlab # [ 205.731747] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.663Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5f6b25f1edafa7f8a6ad3359","created_at":"2026-06-30T07:13:01.850Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d71ff86ab464da169978e8a4cb3114e","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.855Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5f6b25f1edafa7f8a6ad3359: done: 0.027537 sec","job_status":"done","queue_duration_s":0.780074,"scheduling_latency_s":0.780208,"redis_calls":7,"redis_duration_s":0.001726,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001273,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027537,"completed_at":"2026-06-30T07:13:02.663Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006863,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1583vm-test-run-nixbot-gitlab> gitlab # [ 205.832909] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f6b9993c8732b9e0043d7f02","created_at":"2026-06-30T07:13:01.897Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b258158a8fb5d6ca6bac42ceb81a96c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.903Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f6b9993c8732b9e0043d7f02: start","job_status":"start","queue_duration_s":0.761192,"scheduling_latency_s":0.761216}1584vm-test-run-nixbot-gitlab> gitlab # [ 205.834082] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.677Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f6b9993c8732b9e0043d7f02","created_at":"2026-06-30T07:13:01.897Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b258158a8fb5d6ca6bac42ceb81a96c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.903Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f6b9993c8732b9e0043d7f02: done: 0.013485 sec","job_status":"done","queue_duration_s":0.761192,"scheduling_latency_s":0.761216,"redis_calls":8,"redis_duration_s":0.00202,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001311,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013485,"completed_at":"2026-06-30T07:13:02.677Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1585vm-test-run-nixbot-gitlab> gitlab # [ 205.913792] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.679Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"227216f6a7c8a20c4d3857cf","created_at":"2026-06-30T07:13:01.929Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53d7082c8a6434e81cb0fd4e20902a7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.938Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-227216f6a7c8a20c4d3857cf: start","job_status":"start","queue_duration_s":0.741112,"scheduling_latency_s":0.741133}1586vm-test-run-nixbot-gitlab> gitlab # [ 205.916525] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.692Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"227216f6a7c8a20c4d3857cf","created_at":"2026-06-30T07:13:01.929Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53d7082c8a6434e81cb0fd4e20902a7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.938Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-227216f6a7c8a20c4d3857cf: done: 0.012564 sec","job_status":"done","queue_duration_s":0.741112,"scheduling_latency_s":0.741133,"redis_calls":7,"redis_duration_s":0.0018239999999999999,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000485,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012564,"completed_at":"2026-06-30T07:13:02.692Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1587vm-test-run-nixbot-gitlab> gitlab # [ 206.015156] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1f681ddc87c5372aadf4f9df","created_at":"2026-06-30T07:13:01.963Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a84e3b44c9c38d7206987fcb29a7a601","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.966Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1f681ddc87c5372aadf4f9df: start","job_status":"start","queue_duration_s":0.726997,"scheduling_latency_s":0.727017}1588vm-test-run-nixbot-gitlab> gitlab # [ 206.016296] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.702Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1589vm-test-run-nixbot-gitlab> gitlab # [ 206.017167] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.708Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1f681ddc87c5372aadf4f9df","created_at":"2026-06-30T07:13:01.963Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a84e3b44c9c38d7206987fcb29a7a601","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.966Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1f681ddc87c5372aadf4f9df: done: 0.015401 sec","job_status":"done","queue_duration_s":0.726997,"scheduling_latency_s":0.727017,"redis_calls":7,"redis_duration_s":0.002251,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001566,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000685,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015401,"completed_at":"2026-06-30T07:13:02.708Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590vm-test-run-nixbot-gitlab> gitlab # [ 206.053392] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.710Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5bf2db5e3e324673be44735b","created_at":"2026-06-30T07:13:01.970Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38ca3f7dbe821f7379848f3b98ead179","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.972Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5bf2db5e3e324673be44735b: start","job_status":"start","queue_duration_s":0.738219,"scheduling_latency_s":0.738249}1591vm-test-run-nixbot-gitlab> gitlab # [ 206.123670] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.718Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1592vm-test-run-nixbot-gitlab> gitlab # [ 206.125245] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.722Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5bf2db5e3e324673be44735b","created_at":"2026-06-30T07:13:01.970Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38ca3f7dbe821f7379848f3b98ead179","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.972Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5bf2db5e3e324673be44735b: done: 0.01245 sec","job_status":"done","queue_duration_s":0.738219,"scheduling_latency_s":0.738249,"redis_calls":7,"redis_duration_s":0.001529,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001103,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01245,"completed_at":"2026-06-30T07:13:02.722Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1593vm-test-run-nixbot-gitlab> gitlab # [ 206.204212] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3b897fa6e4bc567cf9657538","created_at":"2026-06-30T07:13:01.979Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"854dda253090c04e1ece4ee32a895ff9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.983Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3b897fa6e4bc567cf9657538: start","job_status":"start","queue_duration_s":0.740764,"scheduling_latency_s":0.740786}1594vm-test-run-nixbot-gitlab> gitlab # [ 206.205313] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3b897fa6e4bc567cf9657538","created_at":"2026-06-30T07:13:01.979Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"854dda253090c04e1ece4ee32a895ff9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.983Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3b897fa6e4bc567cf9657538: done: 0.017951 sec","job_status":"done","queue_duration_s":0.740764,"scheduling_latency_s":0.740786,"redis_calls":10,"redis_duration_s":0.002319,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001356,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004330159,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.017951,"completed_at":"2026-06-30T07:13:02.741Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002383,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1595vm-test-run-nixbot-gitlab> gitlab # [ 206.278653] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.743Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"87330b6cbf43780ba3d37bcb","created_at":"2026-06-30T07:13:01.988Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aad903112dde2c5ae5fa05f9d24db7e4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.997Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-87330b6cbf43780ba3d37bcb: start","job_status":"start","queue_duration_s":0.745826,"scheduling_latency_s":0.745847}1596vm-test-run-nixbot-gitlab> gitlab # [ 206.280081] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.768Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"87330b6cbf43780ba3d37bcb","created_at":"2026-06-30T07:13:01.988Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aad903112dde2c5ae5fa05f9d24db7e4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:01.997Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-87330b6cbf43780ba3d37bcb: done: 0.025322 sec","job_status":"done","queue_duration_s":0.745826,"scheduling_latency_s":0.745847,"redis_calls":12,"redis_duration_s":0.003306,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001446,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000502298,"exclusive_lock_hold_duration_s":0.007806021000000001,"duration_s":0.025322,"completed_at":"2026-06-30T07:13:02.768Z","load_balancing_strategy":"primary","db_duration_s":0.004275,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1597vm-test-run-nixbot-gitlab> gitlab # [ 206.350156] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:02.772Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9a9d0185b22878e95e2b5953","created_at":"2026-06-30T07:13:02.001Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"befdf9957d6c9c69ac879effdc451f88","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.003Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9a9d0185b22878e95e2b5953: start","job_status":"start","queue_duration_s":0.768527,"scheduling_latency_s":0.768547}1598vm-test-run-nixbot-gitlab> gitlab # [ 206.351649] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9a9d0185b22878e95e2b5953","created_at":"2026-06-30T07:13:02.001Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"befdf9957d6c9c69ac879effdc451f88","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.003Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9a9d0185b22878e95e2b5953: done: 0.412001 sec","job_status":"done","queue_duration_s":0.768527,"scheduling_latency_s":0.768547,"redis_calls":571,"redis_duration_s":0.136405,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.134722,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.306393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.412001,"completed_at":"2026-06-30T07:13:03.184Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1599vm-test-run-nixbot-gitlab> gitlab # [ 206.409333] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"90604ef45e68f64568819370","created_at":"2026-06-30T07:13:02.017Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f05a3d613e3912ca0a1cd5a3ceefce50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.022Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-90604ef45e68f64568819370: start","job_status":"start","queue_duration_s":1.163287,"scheduling_latency_s":1.163359}1600vm-test-run-nixbot-gitlab> gitlab # [ 206.481597] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"90604ef45e68f64568819370","created_at":"2026-06-30T07:13:02.017Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f05a3d613e3912ca0a1cd5a3ceefce50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.022Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-90604ef45e68f64568819370: done: 0.027707 sec","job_status":"done","queue_duration_s":1.163287,"scheduling_latency_s":1.163359,"redis_calls":9,"redis_duration_s":0.002312,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001205,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000524369,"exclusive_lock_hold_duration_s":0.013598376,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.027707,"completed_at":"2026-06-30T07:13:03.213Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006128,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1601vm-test-run-nixbot-gitlab> gitlab # [ 206.561231] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2ca88e28e0eb68948f419ed9","created_at":"2026-06-30T07:13:02.032Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"930ab9706a1d3fbd4fdb14264c74a523","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.035Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2ca88e28e0eb68948f419ed9: start","job_status":"start","queue_duration_s":1.179448,"scheduling_latency_s":1.17947}1602vm-test-run-nixbot-gitlab> gitlab # [ 206.562852] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.229Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2ca88e28e0eb68948f419ed9","created_at":"2026-06-30T07:13:02.032Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"930ab9706a1d3fbd4fdb14264c74a523","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.035Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2ca88e28e0eb68948f419ed9: done: 0.014599 sec","job_status":"done","queue_duration_s":1.179448,"scheduling_latency_s":1.17947,"redis_calls":9,"redis_duration_s":0.0023230000000000004,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001208,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000501181,"exclusive_lock_hold_duration_s":0.002032102,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.014599,"completed_at":"2026-06-30T07:13:03.229Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1603vm-test-run-nixbot-gitlab> gitlab # [ 206.634191] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8476ff384ed449ac2c022528","created_at":"2026-06-30T07:13:02.044Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63e6212289ee7d4b8d1af2790f5045a3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.047Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8476ff384ed449ac2c022528: start","job_status":"start","queue_duration_s":1.183348,"scheduling_latency_s":1.183369}1604vm-test-run-nixbot-gitlab> gitlab # [ 206.635470] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.243Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8476ff384ed449ac2c022528","created_at":"2026-06-30T07:13:02.044Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63e6212289ee7d4b8d1af2790f5045a3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.047Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8476ff384ed449ac2c022528: done: 0.012762 sec","job_status":"done","queue_duration_s":1.183348,"scheduling_latency_s":1.183369,"redis_calls":7,"redis_duration_s":0.001605,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012762,"completed_at":"2026-06-30T07:13:03.243Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000736,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1605vm-test-run-nixbot-gitlab> gitlab # [ 206.717544] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.244Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"de714469193812ce95fd9acb","created_at":"2026-06-30T07:13:02.055Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d16bc718656a781b3d2c5035dcafe121","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.060Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-de714469193812ce95fd9acb: start","job_status":"start","queue_duration_s":1.183795,"scheduling_latency_s":1.183851}1606vm-test-run-nixbot-gitlab> gitlab # [ 206.719303] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.260Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"de714469193812ce95fd9acb","created_at":"2026-06-30T07:13:02.055Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d16bc718656a781b3d2c5035dcafe121","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.060Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-de714469193812ce95fd9acb: done: 0.016188 sec","job_status":"done","queue_duration_s":1.183795,"scheduling_latency_s":1.183851,"redis_calls":7,"redis_duration_s":0.001609,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001158,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.016188,"completed_at":"2026-06-30T07:13:03.260Z","load_balancing_strategy":"primary","db_duration_s":0.004059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1607vm-test-run-nixbot-gitlab> gitlab # [ 206.762385] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.262Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"138e3b0631b5394d290db4ca","created_at":"2026-06-30T07:13:02.078Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed772fb7ba2064fe55995b170d1368b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-30T07:13:02.082Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-138e3b0631b5394d290db4ca: start","job_status":"start","queue_duration_s":1.179761,"scheduling_latency_s":1.179784}1608vm-test-run-nixbot-gitlab> gitlab # [ 206.838579] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.269Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1609vm-test-run-nixbot-gitlab> gitlab # [ 206.840237] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.273Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"138e3b0631b5394d290db4ca","created_at":"2026-06-30T07:13:02.078Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed772fb7ba2064fe55995b170d1368b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-30T07:13:02.082Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-138e3b0631b5394d290db4ca: done: 0.011728 sec","job_status":"done","queue_duration_s":1.179761,"scheduling_latency_s":1.179784,"redis_calls":7,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011728,"completed_at":"2026-06-30T07:13:03.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610vm-test-run-nixbot-gitlab> gitlab # [ 206.916216] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9ef3584dbab7716e10059ac5","created_at":"2026-06-30T07:13:02.109Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.117Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9ef3584dbab7716e10059ac5: start","job_status":"start","queue_duration_s":1.157595,"scheduling_latency_s":1.157622}1611vm-test-run-nixbot-gitlab> gitlab # [ 206.917475] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.376Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9ef3584dbab7716e10059ac5","created_at":"2026-06-30T07:13:02.109Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.117Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9ef3584dbab7716e10059ac5: done: 0.101689 sec","job_status":"done","queue_duration_s":1.157595,"scheduling_latency_s":1.157622,"redis_calls":60,"redis_duration_s":0.036141,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000925,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.034444,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.101689,"completed_at":"2026-06-30T07:13:03.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1612vm-test-run-nixbot-gitlab> gitlab # [ 207.001957] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"552add478e9c3ebe5e991ee1","created_at":"2026-06-30T07:13:02.137Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50be8cb03a84f38cdfc921c700911f03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.147Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-552add478e9c3ebe5e991ee1: start","job_status":"start","queue_duration_s":1.231298,"scheduling_latency_s":1.231328}1613vm-test-run-nixbot-gitlab> gitlab # [ 207.003237] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.386Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1614vm-test-run-nixbot-gitlab> gitlab # [ 207.003844] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"552add478e9c3ebe5e991ee1","created_at":"2026-06-30T07:13:02.137Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50be8cb03a84f38cdfc921c700911f03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.147Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-552add478e9c3ebe5e991ee1: done: 0.01247 sec","job_status":"done","queue_duration_s":1.231298,"scheduling_latency_s":1.231328,"redis_calls":7,"redis_duration_s":0.001715,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000548,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01247,"completed_at":"2026-06-30T07:13:03.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1615vm-test-run-nixbot-gitlab> gitlab # [ 207.086561] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.392Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b3a80605820bb83b72161acf","created_at":"2026-06-30T07:13:02.171Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95aa7a9662b90aa7acd8bcd8fcdf236e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-30T07:13:02.180Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b3a80605820bb83b72161acf: start","job_status":"start","queue_duration_s":1.211529,"scheduling_latency_s":1.21155}1616vm-test-run-nixbot-gitlab> gitlab # [ 207.088285] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.400Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1617vm-test-run-nixbot-gitlab> gitlab # [ 207.089287] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b3a80605820bb83b72161acf","created_at":"2026-06-30T07:13:02.171Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95aa7a9662b90aa7acd8bcd8fcdf236e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-30T07:13:02.180Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b3a80605820bb83b72161acf: done: 0.012721 sec","job_status":"done","queue_duration_s":1.211529,"scheduling_latency_s":1.21155,"redis_calls":8,"redis_duration_s":0.001926,"redis_read_bytes":133,"redis_write_bytes":867,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000223,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":61,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012721,"completed_at":"2026-06-30T07:13:03.404Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1618vm-test-run-nixbot-gitlab> gitlab # [ 207.161242] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"fd007244273bdb289111a346","created_at":"2026-06-30T07:13:02.204Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b195510ace88744c22b8b4ea57902112","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.214Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-fd007244273bdb289111a346: start","job_status":"start","queue_duration_s":1.191678,"scheduling_latency_s":1.1917}1619vm-test-run-nixbot-gitlab> gitlab # [ 207.162981] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.439Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"fd007244273bdb289111a346","created_at":"2026-06-30T07:13:02.204Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b195510ace88744c22b8b4ea57902112","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.214Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-fd007244273bdb289111a346: done: 0.033564 sec","job_status":"done","queue_duration_s":1.191678,"scheduling_latency_s":1.1917,"redis_calls":14,"redis_duration_s":0.003564,"redis_read_bytes":647,"redis_write_bytes":1515,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.001294,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":346,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001218,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000504534,"exclusive_lock_hold_duration_s":0.00974202,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.033564,"completed_at":"2026-06-30T07:13:03.439Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011152,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620vm-test-run-nixbot-gitlab> gitlab # [ 207.248372] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"c9b504ef8d755fd3429f0106","created_at":"2026-06-30T07:13:02.262Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"831f9f48db37af6dcfe001aa444a9dbd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.271Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-c9b504ef8d755fd3429f0106: start","job_status":"start","queue_duration_s":1.169595,"scheduling_latency_s":1.169615}1621vm-test-run-nixbot-gitlab> gitlab # [ 207.249991] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"c9b504ef8d755fd3429f0106","created_at":"2026-06-30T07:13:02.262Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"831f9f48db37af6dcfe001aa444a9dbd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.271Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-c9b504ef8d755fd3429f0106: done: 0.02684 sec","job_status":"done","queue_duration_s":1.169595,"scheduling_latency_s":1.169615,"redis_calls":9,"redis_duration_s":0.006413,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003315,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000462349,"exclusive_lock_hold_duration_s":0.008482084,"duration_s":0.02684,"completed_at":"2026-06-30T07:13:03.468Z","load_balancing_strategy":"primary","db_duration_s":0.007095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1622vm-test-run-nixbot-gitlab> gitlab # [ 207.319308] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fd4992df67e3135a5482ebed","created_at":"2026-06-30T07:13:02.288Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e13e2e4d0e52a4e18add12e0b39e55af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-30T07:13:02.291Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fd4992df67e3135a5482ebed: start","job_status":"start","queue_duration_s":1.177693,"scheduling_latency_s":1.177716}1623vm-test-run-nixbot-gitlab> gitlab # [ 207.321641] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.487Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fd4992df67e3135a5482ebed","created_at":"2026-06-30T07:13:02.288Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-a382f13b446c4153","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e13e2e4d0e52a4e18add12e0b39e55af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-30T07:13:02.291Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fd4992df67e3135a5482ebed: done: 0.018219 sec","job_status":"done","queue_duration_s":1.177693,"scheduling_latency_s":1.177716,"redis_calls":7,"redis_duration_s":0.001833,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000599,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018219,"completed_at":"2026-06-30T07:13:03.487Z","load_balancing_strategy":"primary","db_duration_s":0.001175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1624vm-test-run-nixbot-gitlab> gitlab # [ 207.404400] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.489Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f6a698929f31af2aed447c34","created_at":"2026-06-30T07:13:02.604Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-d9efd306f8b74bec","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.622Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f6a698929f31af2aed447c34: start","job_status":"start","queue_duration_s":0.866307,"scheduling_latency_s":0.86633}1625vm-test-run-nixbot-gitlab> gitlab # [ 207.406165] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.503Z","message":"No existing merge request to be cleaned up."}1626vm-test-run-nixbot-gitlab> gitlab # [ 207.407146] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.509Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f6a698929f31af2aed447c34","created_at":"2026-06-30T07:13:02.604Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-d9efd306f8b74bec","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.622Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f6a698929f31af2aed447c34: done: 0.020463 sec","job_status":"done","queue_duration_s":0.866307,"scheduling_latency_s":0.86633,"redis_calls":10,"redis_duration_s":0.006786,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00507,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020463,"completed_at":"2026-06-30T07:13:03.509Z","load_balancing_strategy":"primary","db_duration_s":0.001728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1627vm-test-run-nixbot-gitlab> gitlab # [ 207.481380] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.511Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"02f5921a230e403d93502362","created_at":"2026-06-30T07:13:02.604Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-d9efd306f8b74bec","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.622Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-02f5921a230e403d93502362: start","job_status":"start","queue_duration_s":0.888493,"scheduling_latency_s":0.888516}1628vm-test-run-nixbot-gitlab> gitlab # [ 207.486239] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.520Z","message":"No existing merge request to be cleaned up."}1629vm-test-run-nixbot-gitlab> gitlab # [ 207.488371] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.526Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"02f5921a230e403d93502362","created_at":"2026-06-30T07:13:02.604Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-d9efd306f8b74bec","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.622Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-02f5921a230e403d93502362: done: 0.015256 sec","job_status":"done","queue_duration_s":0.888493,"scheduling_latency_s":0.888516,"redis_calls":10,"redis_duration_s":0.002266,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001358,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015256,"completed_at":"2026-06-30T07:13:03.526Z","load_balancing_strategy":"primary","db_duration_s":0.001532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1630vm-test-run-nixbot-gitlab> gitlab # [ 207.550204] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.527Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0cd8800d2a2393c1ee357849","created_at":"2026-06-30T07:13:02.604Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-d9efd306f8b74bec","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.622Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0cd8800d2a2393c1ee357849: start","job_status":"start","queue_duration_s":0.905095,"scheduling_latency_s":0.905116}1631vm-test-run-nixbot-gitlab> gitlab # [ 207.570076] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.537Z","message":"No existing merge request to be cleaned up."}1632vm-test-run-nixbot-gitlab> gitlab # [ 207.573098] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.543Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0cd8800d2a2393c1ee357849","created_at":"2026-06-30T07:13:02.604Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-d9efd306f8b74bec","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.622Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0cd8800d2a2393c1ee357849: done: 0.015311 sec","job_status":"done","queue_duration_s":0.905095,"scheduling_latency_s":0.905116,"redis_calls":10,"redis_duration_s":0.002332,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001459,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015311,"completed_at":"2026-06-30T07:13:03.543Z","load_balancing_strategy":"primary","db_duration_s":0.001509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1633vm-test-run-nixbot-gitlab> gitlab # [ 207.644294] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.544Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"01e6e91331415e0ed77031d4","created_at":"2026-06-30T07:13:02.604Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-d9efd306f8b74bec","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.622Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-01e6e91331415e0ed77031d4: start","job_status":"start","queue_duration_s":0.921698,"scheduling_latency_s":0.92172}1634vm-test-run-nixbot-gitlab> gitlab # [ 207.652220] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.561Z","message":"No existing merge request to be cleaned up."}1635vm-test-run-nixbot-gitlab> gitlab # [ 207.655108] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.571Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"01e6e91331415e0ed77031d4","created_at":"2026-06-30T07:13:02.604Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-d9efd306f8b74bec","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a15985de1dab65b2aa8d66800967b056","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:02.622Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-01e6e91331415e0ed77031d4: done: 0.027143 sec","job_status":"done","queue_duration_s":0.921698,"scheduling_latency_s":0.92172,"redis_calls":10,"redis_duration_s":0.005981,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001388,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027143,"completed_at":"2026-06-30T07:13:03.571Z","load_balancing_strategy":"primary","db_duration_s":0.001625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1636vm-test-run-nixbot-gitlab> gitlab # [ 207.717139] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb6f38627ce6bc6603523492","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb6f38627ce6bc6603523492: start","job_status":"start","queue_duration_s":0.205351,"scheduling_latency_s":0.205376}1637vm-test-run-nixbot-gitlab> gitlab # [ 207.788806] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb6f38627ce6bc6603523492","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb6f38627ce6bc6603523492: done: 0.01797 sec","job_status":"done","queue_duration_s":0.205351,"scheduling_latency_s":0.205376,"redis_calls":15,"redis_duration_s":0.004266,"redis_read_bytes":266,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":132,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002205,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01797,"completed_at":"2026-06-30T07:13:03.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638vm-test-run-nixbot-gitlab> gitlab # [ 207.800107] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"166988a6617a7a3cb08a3d12","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-166988a6617a7a3cb08a3d12: start","job_status":"start","queue_duration_s":0.224609,"scheduling_latency_s":0.224632}1639vm-test-run-nixbot-gitlab> gitlab # [ 207.819869] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"166988a6617a7a3cb08a3d12","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-166988a6617a7a3cb08a3d12: done: 0.015634 sec","job_status":"done","queue_duration_s":0.224609,"scheduling_latency_s":0.224632,"redis_calls":13,"redis_duration_s":0.002882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015634,"completed_at":"2026-06-30T07:13:03.610Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1640vm-test-run-nixbot-gitlab> gitlab # [ 207.894452] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6eb7ec6298ba2262fb3afbf9","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6eb7ec6298ba2262fb3afbf9: start","job_status":"start","queue_duration_s":0.241785,"scheduling_latency_s":0.241806}1641vm-test-run-nixbot-gitlab> gitlab # [ 207.900974] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6eb7ec6298ba2262fb3afbf9","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6eb7ec6298ba2262fb3afbf9: done: 0.014971 sec","job_status":"done","queue_duration_s":0.241785,"scheduling_latency_s":0.241806,"redis_calls":13,"redis_duration_s":0.002887,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014971,"completed_at":"2026-06-30T07:13:03.626Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1642vm-test-run-nixbot-gitlab> gitlab # [ 208.028138] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3c56a4d4d38c30bf1b00ebd","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3c56a4d4d38c30bf1b00ebd: start","job_status":"start","queue_duration_s":0.258087,"scheduling_latency_s":0.258109}1643vm-test-run-nixbot-gitlab> gitlab # [ 208.029426] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3c56a4d4d38c30bf1b00ebd","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3c56a4d4d38c30bf1b00ebd: done: 0.016554 sec","job_status":"done","queue_duration_s":0.258087,"scheduling_latency_s":0.258109,"redis_calls":13,"redis_duration_s":0.004419,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002419,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016554,"completed_at":"2026-06-30T07:13:03.644Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1644vm-test-run-nixbot-gitlab> gitlab # [ 208.103952] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2240c91356fe05f551447400","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2240c91356fe05f551447400: start","job_status":"start","queue_duration_s":0.275993,"scheduling_latency_s":0.276016}1645vm-test-run-nixbot-gitlab> gitlab # [ 208.105408] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2240c91356fe05f551447400","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2240c91356fe05f551447400: done: 0.024036 sec","job_status":"done","queue_duration_s":0.275993,"scheduling_latency_s":0.276016,"redis_calls":13,"redis_duration_s":0.011684,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024036,"completed_at":"2026-06-30T07:13:03.669Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1646vm-test-run-nixbot-gitlab> gitlab # [ 208.175094] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90f174b3e6c1bc93a58a7174","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90f174b3e6c1bc93a58a7174: start","job_status":"start","queue_duration_s":0.30322,"scheduling_latency_s":0.303245}1647vm-test-run-nixbot-gitlab> gitlab # [ 208.176375] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90f174b3e6c1bc93a58a7174","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90f174b3e6c1bc93a58a7174: done: 0.016763 sec","job_status":"done","queue_duration_s":0.30322,"scheduling_latency_s":0.303245,"redis_calls":13,"redis_duration_s":0.004044,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001659,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002385,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016763,"completed_at":"2026-06-30T07:13:03.689Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1648vm-test-run-nixbot-gitlab> gitlab # [ 208.252061] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"802191bfd01786b2cd064c30","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-802191bfd01786b2cd064c30: start","job_status":"start","queue_duration_s":0.321301,"scheduling_latency_s":0.321332}1649vm-test-run-nixbot-gitlab> gitlab # [ 208.253403] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"802191bfd01786b2cd064c30","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-802191bfd01786b2cd064c30: done: 0.015473 sec","job_status":"done","queue_duration_s":0.321301,"scheduling_latency_s":0.321332,"redis_calls":13,"redis_duration_s":0.003308,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001145,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015473,"completed_at":"2026-06-30T07:13:03.706Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1650vm-test-run-nixbot-gitlab> gitlab # [ 208.286236] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fad3ed187747e3447f87dcc","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fad3ed187747e3447f87dcc: start","job_status":"start","queue_duration_s":0.338154,"scheduling_latency_s":0.338175}1651vm-test-run-nixbot-gitlab> gitlab # [ 208.355210] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fad3ed187747e3447f87dcc","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fad3ed187747e3447f87dcc: done: 0.015549 sec","job_status":"done","queue_duration_s":0.338154,"scheduling_latency_s":0.338175,"redis_calls":13,"redis_duration_s":0.003045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015549,"completed_at":"2026-06-30T07:13:03.723Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1652vm-test-run-nixbot-gitlab> gitlab # [ 208.366942] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"704bc3e81d4825303ba35df1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-704bc3e81d4825303ba35df1: start","job_status":"start","queue_duration_s":0.355069,"scheduling_latency_s":0.35509}1653vm-test-run-nixbot-gitlab> gitlab # [ 208.387160] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"704bc3e81d4825303ba35df1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-704bc3e81d4825303ba35df1: done: 0.015347 sec","job_status":"done","queue_duration_s":0.355069,"scheduling_latency_s":0.35509,"redis_calls":13,"redis_duration_s":0.003248,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002314,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015347,"completed_at":"2026-06-30T07:13:03.740Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1654vm-test-run-nixbot-gitlab> gitlab # [ 208.457259] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a37c358fce6508f661778c3b","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a37c358fce6508f661778c3b: start","job_status":"start","queue_duration_s":0.371711,"scheduling_latency_s":0.371734}1655vm-test-run-nixbot-gitlab> gitlab # [ 208.526589] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a37c358fce6508f661778c3b","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a37c358fce6508f661778c3b: done: 0.01517 sec","job_status":"done","queue_duration_s":0.371711,"scheduling_latency_s":0.371734,"redis_calls":13,"redis_duration_s":0.002981,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01517,"completed_at":"2026-06-30T07:13:03.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1656vm-test-run-nixbot-gitlab> gitlab # [ 208.529264] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"007e987e6274f4c5c747b841","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-007e987e6274f4c5c747b841: start","job_status":"start","queue_duration_s":0.388223,"scheduling_latency_s":0.388245}1657vm-test-run-nixbot-gitlab> gitlab # [ 208.549174] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"007e987e6274f4c5c747b841","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-007e987e6274f4c5c747b841: done: 0.015062 sec","job_status":"done","queue_duration_s":0.388223,"scheduling_latency_s":0.388245,"redis_calls":13,"redis_duration_s":0.0031320000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015062,"completed_at":"2026-06-30T07:13:03.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1658vm-test-run-nixbot-gitlab> gitlab # [ 208.647087] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e68cc1c58d60de2be2fd847","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e68cc1c58d60de2be2fd847: start","job_status":"start","queue_duration_s":0.404616,"scheduling_latency_s":0.404637}1659vm-test-run-nixbot-gitlab> gitlab # [ 208.648460] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e68cc1c58d60de2be2fd847","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e68cc1c58d60de2be2fd847: done: 0.016017 sec","job_status":"done","queue_duration_s":0.404616,"scheduling_latency_s":0.404637,"redis_calls":13,"redis_duration_s":0.003856,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002802,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016017,"completed_at":"2026-06-30T07:13:03.790Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1660vm-test-run-nixbot-gitlab> gitlab # [ 208.692400] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6edbc825c19779c8005c3b9c","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6edbc825c19779c8005c3b9c: start","job_status":"start","queue_duration_s":0.421982,"scheduling_latency_s":0.422003}1661vm-test-run-nixbot-gitlab> gitlab # [ 208.712563] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6edbc825c19779c8005c3b9c","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6edbc825c19779c8005c3b9c: done: 0.014874 sec","job_status":"done","queue_duration_s":0.421982,"scheduling_latency_s":0.422003,"redis_calls":13,"redis_duration_s":0.002925,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014874,"completed_at":"2026-06-30T07:13:03.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1662vm-test-run-nixbot-gitlab> gitlab # [ 208.792526] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcd242ae69f0f813d6772ed1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcd242ae69f0f813d6772ed1: start","job_status":"start","queue_duration_s":0.438163,"scheduling_latency_s":0.438186}1663vm-test-run-nixbot-gitlab> gitlab # [ 208.794569] setup-gitlab-start[3023]: ++ jq -r .token1664vm-test-run-nixbot-gitlab> gitlab # [ 208.795533] runuser[2768]: pam_unix(runuser:session): session closed for user gitlab1665vm-test-run-nixbot-gitlab> gitlab # [ 208.796876] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcd242ae69f0f813d6772ed1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcd242ae69f0f813d6772ed1: done: 0.015156 sec","job_status":"done","queue_duration_s":0.438163,"scheduling_latency_s":0.438186,"redis_calls":13,"redis_duration_s":0.003147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002242,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015156,"completed_at":"2026-06-30T07:13:03.823Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1666vm-test-run-nixbot-gitlab> gitlab # [ 208.860168] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f3d68ceb259fbd2ad915dbe","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f3d68ceb259fbd2ad915dbe: start","job_status":"start","queue_duration_s":0.45462,"scheduling_latency_s":0.454643}1667vm-test-run-nixbot-gitlab> gitlab # [ 208.936278] setup-gitlab-start[3024]: +++ date -d '+7 days' +%F1668vm-test-run-nixbot-gitlab> gitlab # [ 208.937969] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f3d68ceb259fbd2ad915dbe","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f3d68ceb259fbd2ad915dbe: done: 0.014939 sec","job_status":"done","queue_duration_s":0.45462,"scheduling_latency_s":0.454643,"redis_calls":13,"redis_duration_s":0.002894,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009927,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014939,"completed_at":"2026-06-30T07:13:03.839Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1669vm-test-run-nixbot-gitlab> gitlab # [ 208.942099] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"347a7dcf2e3e3f5b55de25fc","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-347a7dcf2e3e3f5b55de25fc: start","job_status":"start","queue_duration_s":0.470832,"scheduling_latency_s":0.470897}1670vm-test-run-nixbot-gitlab> gitlab # [ 209.011625] setup-gitlab-start[3022]: ++ curl -fs -X POST -H 'Authorization: Bearer 8474ca25eb314ff09be742be92b15190486076877f30c008c1ba20e4b26e4ece' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-07"}' http://gitlab/api/v4/users/1/personal_access_tokens1671vm-test-run-nixbot-gitlab> gitlab # [ 209.013430] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:05 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 2401672vm-test-run-nixbot-gitlab> gitlab # [ 209.014222] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"347a7dcf2e3e3f5b55de25fc","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-347a7dcf2e3e3f5b55de25fc: done: 0.019254 sec","job_status":"done","queue_duration_s":0.470832,"scheduling_latency_s":0.470897,"redis_calls":13,"redis_duration_s":0.0070799999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00605,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019254,"completed_at":"2026-06-30T07:13:03.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1673vm-test-run-nixbot-gitlab> gitlab # [ 209.088250] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3aa471835731aefcbc13afa9","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3aa471835731aefcbc13afa9: start","job_status":"start","queue_duration_s":0.491435,"scheduling_latency_s":0.491456}1674vm-test-run-nixbot-gitlab> gitlab # [ 209.090187] setup-gitlab-start[2717]: + TOKEN=glpat-GVdrDLp71PSu47q7_3EzPm86MQp1OjEH.01.0w1qi4zpu1675vm-test-run-nixbot-gitlab> gitlab # [ 209.091191] setup-gitlab-start[2717]: + test -n glpat-GVdrDLp71PSu47q7_3EzPm86MQp1OjEH.01.0w1qi4zpu1676vm-test-run-nixbot-gitlab> gitlab # [ 209.091987] setup-gitlab-start[2717]: + test glpat-GVdrDLp71PSu47q7_3EzPm86MQp1OjEH.01.0w1qi4zpu '!=' null1677vm-test-run-nixbot-gitlab> gitlab # [ 209.092882] setup-gitlab-start[2717]: + mkdir -p /var/lib/secrets1678vm-test-run-nixbot-gitlab> gitlab # [ 209.094005] setup-gitlab-start[2717]: + echo glpat-GVdrDLp71PSu47q7_3EzPm86MQp1OjEH.01.0w1qi4zpu1679vm-test-run-nixbot-gitlab> gitlab # [ 209.094905] setup-gitlab-start[2717]: + chmod 644 /var/lib/secrets/gitlab-token1680vm-test-run-nixbot-gitlab> gitlab # [ 209.095751] setup-gitlab-start[2717]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-GVdrDLp71PSu47q7_3EzPm86MQp1OjEH.01.0w1qi4zpu' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1681vm-test-run-nixbot-gitlab> gitlab # [ 209.096955] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3aa471835731aefcbc13afa9","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3aa471835731aefcbc13afa9: done: 0.015266 sec","job_status":"done","queue_duration_s":0.491435,"scheduling_latency_s":0.491456,"redis_calls":13,"redis_duration_s":0.003165,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015266,"completed_at":"2026-06-30T07:13:03.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1682vm-test-run-nixbot-gitlab> gitlab # [ 209.165880] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6488d49f1b9031a8dcd2da4e","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6488d49f1b9031a8dcd2da4e: start","job_status":"start","queue_duration_s":0.508034,"scheduling_latency_s":0.508055}1683vm-test-run-nixbot-gitlab> gitlab # [ 209.167231] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6488d49f1b9031a8dcd2da4e","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6488d49f1b9031a8dcd2da4e: done: 0.014942 sec","job_status":"done","queue_duration_s":0.508034,"scheduling_latency_s":0.508055,"redis_calls":13,"redis_duration_s":0.002895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014942,"completed_at":"2026-06-30T07:13:03.892Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1684vm-test-run-nixbot-gitlab> gitlab # [ 209.239373] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"616532e659b2e5138f08ecec","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-616532e659b2e5138f08ecec: start","job_status":"start","queue_duration_s":0.524278,"scheduling_latency_s":0.5243}1685vm-test-run-nixbot-gitlab> gitlab # [ 209.240575] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"616532e659b2e5138f08ecec","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-616532e659b2e5138f08ecec: done: 0.01485 sec","job_status":"done","queue_duration_s":0.524278,"scheduling_latency_s":0.5243,"redis_calls":13,"redis_duration_s":0.002979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01485,"completed_at":"2026-06-30T07:13:03.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1686vm-test-run-nixbot-gitlab> gitlab # [ 209.313293] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7cac9dd4d66ead3bc8aa926","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7cac9dd4d66ead3bc8aa926: start","job_status":"start","queue_duration_s":0.540419,"scheduling_latency_s":0.540441}1687vm-test-run-nixbot-gitlab> gitlab # [ 209.315049] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7cac9dd4d66ead3bc8aa926","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7cac9dd4d66ead3bc8aa926: done: 0.014845 sec","job_status":"done","queue_duration_s":0.540419,"scheduling_latency_s":0.540441,"redis_calls":13,"redis_duration_s":0.002916,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014845,"completed_at":"2026-06-30T07:13:03.925Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1688vm-test-run-nixbot-gitlab> gitlab # [ 209.386189] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88cac9f24c2da016e8107a4f","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88cac9f24c2da016e8107a4f: start","job_status":"start","queue_duration_s":0.556556,"scheduling_latency_s":0.556579}1689vm-test-run-nixbot-gitlab> gitlab # [ 209.388128] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88cac9f24c2da016e8107a4f","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88cac9f24c2da016e8107a4f: done: 0.015051 sec","job_status":"done","queue_duration_s":0.556556,"scheduling_latency_s":0.556579,"redis_calls":13,"redis_duration_s":0.003077,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015051,"completed_at":"2026-06-30T07:13:03.941Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1690vm-test-run-nixbot-gitlab> gitlab # [ 209.465159] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e23602cf24c1cb07c61c1173","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e23602cf24c1cb07c61c1173: start","job_status":"start","queue_duration_s":0.572967,"scheduling_latency_s":0.572988}1691vm-test-run-nixbot-gitlab> gitlab # [ 209.535363] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e23602cf24c1cb07c61c1173","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e23602cf24c1cb07c61c1173: done: 0.015006 sec","job_status":"done","queue_duration_s":0.572967,"scheduling_latency_s":0.572988,"redis_calls":13,"redis_duration_s":0.003035,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015006,"completed_at":"2026-06-30T07:13:03.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1692vm-test-run-nixbot-gitlab> gitlab # [ 209.537076] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ed1a534907698939333f37f","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ed1a534907698939333f37f: start","job_status":"start","queue_duration_s":0.589256,"scheduling_latency_s":0.589277}1693vm-test-run-nixbot-gitlab> gitlab # [ 209.616366] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ed1a534907698939333f37f","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ed1a534907698939333f37f: done: 0.017326 sec","job_status":"done","queue_duration_s":0.589256,"scheduling_latency_s":0.589277,"redis_calls":13,"redis_duration_s":0.003247,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017326,"completed_at":"2026-06-30T07:13:03.976Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1694vm-test-run-nixbot-gitlab> gitlab # [ 209.618218] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eab43cee5a7dd8cd8542b767","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eab43cee5a7dd8cd8542b767: start","job_status":"start","queue_duration_s":0.608188,"scheduling_latency_s":0.60822}1695vm-test-run-nixbot-gitlab> gitlab # [ 209.697846] gitaly[1372]: time="2026-06-30T07:13:06.491Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.490" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1696vm-test-run-nixbot-gitlab> gitlab # [ 209.699369] gitaly[1372]: time="2026-06-30T07:13:06.491Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.490" grpc.time_ms=0.388 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1697vm-test-run-nixbot-gitlab> gitlab # [ 209.700283] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eab43cee5a7dd8cd8542b767","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eab43cee5a7dd8cd8542b767: done: 0.016053 sec","job_status":"done","queue_duration_s":0.608188,"scheduling_latency_s":0.60822,"redis_calls":13,"redis_duration_s":0.003047,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016053,"completed_at":"2026-06-30T07:13:03.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1698vm-test-run-nixbot-gitlab> gitlab # [ 209.774366] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:03.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09ae60ca7dc98530abe62ed1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09ae60ca7dc98530abe62ed1: start","job_status":"start","queue_duration_s":0.625508,"scheduling_latency_s":0.625531}1699vm-test-run-nixbot-gitlab> gitlab # [ 209.776061] gitaly[1372]: time="2026-06-30T07:13:06.572Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.091" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.572" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1700vm-test-run-nixbot-gitlab> gitlab # [ 209.777085] gitaly[1372]: time="2026-06-30T07:13:06.572Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.091" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.572" grpc.time_ms=0.288 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1701vm-test-run-nixbot-gitlab> gitlab # [ 209.786164] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09ae60ca7dc98530abe62ed1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09ae60ca7dc98530abe62ed1: done: 0.015044 sec","job_status":"done","queue_duration_s":0.625508,"scheduling_latency_s":0.625531,"redis_calls":13,"redis_duration_s":0.002969,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015044,"completed_at":"2026-06-30T07:13:04.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1702vm-test-run-nixbot-gitlab> gitlab # [ 209.860356] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"446a8e1359fa3477eb87a4fe","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-446a8e1359fa3477eb87a4fe: start","job_status":"start","queue_duration_s":0.641835,"scheduling_latency_s":0.641892}1703vm-test-run-nixbot-gitlab> gitlab # [ 209.868204] gitaly[1372]: time="2026-06-30T07:13:06.604Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG diskcache=b393dfad-f0a7-4af2-8656-c4b6e7a03945 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-30T07:13:16.097" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.577" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1704vm-test-run-nixbot-gitlab> gitlab # [ 209.942392] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"446a8e1359fa3477eb87a4fe","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-446a8e1359fa3477eb87a4fe: done: 0.01644 sec","job_status":"done","queue_duration_s":0.641835,"scheduling_latency_s":0.641892,"redis_calls":13,"redis_duration_s":0.004125,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01644,"completed_at":"2026-06-30T07:13:04.028Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1705vm-test-run-nixbot-gitlab> gitlab # [ 209.946208] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b1051e610beb26639d22f36","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b1051e610beb26639d22f36: start","job_status":"start","queue_duration_s":0.659584,"scheduling_latency_s":0.659605}1706vm-test-run-nixbot-gitlab> gitlab # [ 210.033949] gitaly[1372]: time="2026-06-30T07:13:06.605Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=323 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-30T07:13:16.097" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.577" grpc.time_ms=27.221 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1707vm-test-run-nixbot-gitlab> gitlab # [ 210.035282] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b1051e610beb26639d22f36","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b1051e610beb26639d22f36: done: 0.014977 sec","job_status":"done","queue_duration_s":0.659584,"scheduling_latency_s":0.659605,"redis_calls":13,"redis_duration_s":0.0028770000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014977,"completed_at":"2026-06-30T07:13:04.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1708vm-test-run-nixbot-gitlab> gitlab # [ 210.115399] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b668510505715a481a5c7803","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b668510505715a481a5c7803: start","job_status":"start","queue_duration_s":0.675836,"scheduling_latency_s":0.675893}1709vm-test-run-nixbot-gitlab> gitlab # [ 210.117091] gitaly[1372]: time="2026-06-30T07:13:06.616Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.135" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.616" pid=1372 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1710vm-test-run-nixbot-gitlab> gitlab # [ 210.118067] gitaly[1372]: time="2026-06-30T07:13:06.617Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.135" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.616" grpc.time_ms=0.75 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1711vm-test-run-nixbot-gitlab> gitlab # [ 210.118976] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b668510505715a481a5c7803","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b668510505715a481a5c7803: done: 0.014959 sec","job_status":"done","queue_duration_s":0.675836,"scheduling_latency_s":0.675893,"redis_calls":13,"redis_duration_s":0.002898,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014959,"completed_at":"2026-06-30T07:13:04.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1712vm-test-run-nixbot-gitlab> gitlab # [ 210.165637] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12cf6eeb3694fc7cab1fc64a","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12cf6eeb3694fc7cab1fc64a: start","job_status":"start","queue_duration_s":0.692125,"scheduling_latency_s":0.692147}1713vm-test-run-nixbot-gitlab> gitlab # [ 210.237212] gitaly[1372]: time="2026-06-30T07:13:06.627Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-30T07:13:16.139" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.620" grpc.time_ms=6.787 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1714vm-test-run-nixbot-gitlab> gitlab # [ 210.239177] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12cf6eeb3694fc7cab1fc64a","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12cf6eeb3694fc7cab1fc64a: done: 0.014823 sec","job_status":"done","queue_duration_s":0.692125,"scheduling_latency_s":0.692147,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014823,"completed_at":"2026-06-30T07:13:04.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1715vm-test-run-nixbot-gitlab> gitlab # [ 210.333250] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"720f3ee4839b5730c43042ba","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-720f3ee4839b5730c43042ba: start","job_status":"start","queue_duration_s":0.708218,"scheduling_latency_s":0.70824}1716vm-test-run-nixbot-gitlab> gitlab # [ 210.334866] gitaly[1372]: time="2026-06-30T07:13:06.664Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.183" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.664" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1717vm-test-run-nixbot-gitlab> gitlab # [ 210.335933] gitaly[1372]: time="2026-06-30T07:13:06.664Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.183" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.664" grpc.time_ms=0.288 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1718vm-test-run-nixbot-gitlab> gitlab # [ 210.337245] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"720f3ee4839b5730c43042ba","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-720f3ee4839b5730c43042ba: done: 0.014756 sec","job_status":"done","queue_duration_s":0.708218,"scheduling_latency_s":0.70824,"redis_calls":13,"redis_duration_s":0.0028710000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014756,"completed_at":"2026-06-30T07:13:04.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1719vm-test-run-nixbot-gitlab> gitlab # [ 210.414524] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3550ae7e371d18c9437314ed","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3550ae7e371d18c9437314ed: start","job_status":"start","queue_duration_s":0.724357,"scheduling_latency_s":0.724379}1720vm-test-run-nixbot-gitlab> gitlab # [ 210.416218] gitaly[1372]: time="2026-06-30T07:13:06.695Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG diskcache=f2a4e893-1c6a-49e8-889d-a6be1a7ee41f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-30T07:13:16.186" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.667" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1721vm-test-run-nixbot-gitlab> gitlab # [ 210.420154] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3550ae7e371d18c9437314ed","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3550ae7e371d18c9437314ed: done: 0.015004 sec","job_status":"done","queue_duration_s":0.724357,"scheduling_latency_s":0.724379,"redis_calls":13,"redis_duration_s":0.003017,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015004,"completed_at":"2026-06-30T07:13:04.109Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1722vm-test-run-nixbot-gitlab> gitlab # [ 210.523906] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c49b9b10f74a9f5c2010cbf8","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c49b9b10f74a9f5c2010cbf8: start","job_status":"start","queue_duration_s":0.740661,"scheduling_latency_s":0.740682}1723vm-test-run-nixbot-gitlab> gitlab # [ 210.525245] gitaly[1372]: time="2026-06-30T07:13:06.696Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-30T07:13:16.186" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.667" grpc.time_ms=28.854 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1724vm-test-run-nixbot-gitlab> gitlab # [ 210.526489] setup-gitlab-start[2717]: + mkdir -p /tmp/test-flake1725vm-test-run-nixbot-gitlab> gitlab # [ 210.527352] setup-gitlab-start[2717]: + cd /tmp/test-flake1726vm-test-run-nixbot-gitlab> gitlab # [ 210.529103] setup-gitlab-start[2717]: + git init -b master1727vm-test-run-nixbot-gitlab> gitlab # [ 210.530113] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c49b9b10f74a9f5c2010cbf8","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c49b9b10f74a9f5c2010cbf8: done: 0.01489 sec","job_status":"done","queue_duration_s":0.740661,"scheduling_latency_s":0.740682,"redis_calls":13,"redis_duration_s":0.0029300000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01489,"completed_at":"2026-06-30T07:13:04.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1728vm-test-run-nixbot-gitlab> gitlab # [ 210.607304] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40b44a8c13358e88dce4aa56","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40b44a8c13358e88dce4aa56: start","job_status":"start","queue_duration_s":0.756841,"scheduling_latency_s":0.756906}1729vm-test-run-nixbot-gitlab> gitlab # [ 210.680470] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:07 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 14951730vm-test-run-nixbot-gitlab> gitlab # [ 210.681760] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:07 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 321731vm-test-run-nixbot-gitlab> gitlab # [ 210.682922] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:07 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1011732vm-test-run-nixbot-gitlab> gitlab # [ 210.684166] gitaly[1372]: time="2026-06-30T07:13:06.702Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.221" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.702" pid=1372 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1733vm-test-run-nixbot-gitlab> gitlab # [ 210.685053] gitaly[1372]: time="2026-06-30T07:13:06.703Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.221" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.702" grpc.time_ms=0.535 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1734vm-test-run-nixbot-gitlab> gitlab # [ 210.686127] setup-gitlab-start[3059]: Initialized empty Git repository in /tmp/test-flake/.git/1735vm-test-run-nixbot-gitlab> gitlab # [ 210.687145] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40b44a8c13358e88dce4aa56","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40b44a8c13358e88dce4aa56: done: 0.014795 sec","job_status":"done","queue_duration_s":0.756841,"scheduling_latency_s":0.756906,"redis_calls":13,"redis_duration_s":0.002894,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014795,"completed_at":"2026-06-30T07:13:04.141Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1736vm-test-run-nixbot-gitlab> gitlab # [ 210.790164] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5dead39b86f867ca9606c23","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5dead39b86f867ca9606c23: start","job_status":"start","queue_duration_s":0.772945,"scheduling_latency_s":0.772966}1737vm-test-run-nixbot-gitlab> gitlab # [ 210.792212] gitaly[1372]: time="2026-06-30T07:13:06.715Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-30T07:13:16.225" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.706" grpc.time_ms=8.856 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1738vm-test-run-nixbot-gitlab> gitlab # [ 210.793285] setup-gitlab-start[2717]: + git config user.name test1739vm-test-run-nixbot-gitlab> gitlab # [ 210.794088] setup-gitlab-start[2717]: + git config user.email test@example.com1740vm-test-run-nixbot-gitlab> gitlab # [ 210.794924] setup-gitlab-start[2717]: + cat1741vm-test-run-nixbot-gitlab> gitlab # [ 210.796113] setup-gitlab-start[2717]: + git add flake.nix1742vm-test-run-nixbot-gitlab> gitlab # [ 210.796979] setup-gitlab-start[2717]: + git commit -m 'initial commit'1743vm-test-run-nixbot-gitlab> gitlab # [ 210.798221] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5dead39b86f867ca9606c23","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5dead39b86f867ca9606c23: done: 0.014827 sec","job_status":"done","queue_duration_s":0.772945,"scheduling_latency_s":0.772966,"redis_calls":13,"redis_duration_s":0.002934,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014827,"completed_at":"2026-06-30T07:13:04.157Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1744vm-test-run-nixbot-gitlab> gitlab # [ 210.875665] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab086d4203e355d1a0ab22c9","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab086d4203e355d1a0ab22c9: start","job_status":"start","queue_duration_s":0.789104,"scheduling_latency_s":0.789126}1745vm-test-run-nixbot-gitlab> gitlab # [ 210.877152] gitaly[1372]: time="2026-06-30T07:13:06.747Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:16.265" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.746" grpc.time_ms=0.274 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1746vm-test-run-nixbot-gitlab> gitlab # [ 210.877984] gitaly[1372]: time="2026-06-30T07:13:06.754Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-30T07:13:16.272" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.752" grpc.time_ms=0.836 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1747vm-test-run-nixbot-gitlab> gitlab # [ 210.888406] setup-gitlab-start[3064]: [master (root-commit) e6cf5e8] initial commit1748vm-test-run-nixbot-gitlab> gitlab # [ 210.889879] setup-gitlab-start[3064]: 1 file changed, 10 insertions(+)1749vm-test-run-nixbot-gitlab> gitlab # [ 210.891094] setup-gitlab-start[3064]: create mode 100644 flake.nix1750vm-test-run-nixbot-gitlab> gitlab # [ 210.892293] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab086d4203e355d1a0ab22c9","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab086d4203e355d1a0ab22c9: done: 0.014758 sec","job_status":"done","queue_duration_s":0.789104,"scheduling_latency_s":0.789126,"redis_calls":13,"redis_duration_s":0.002839,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014758,"completed_at":"2026-06-30T07:13:04.173Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1751vm-test-run-nixbot-gitlab> gitlab # [ 210.965910] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccf10e42521097efcc9743cf","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccf10e42521097efcc9743cf: start","job_status":"start","queue_duration_s":0.805162,"scheduling_latency_s":0.805183}1752vm-test-run-nixbot-gitlab> gitlab # [ 210.976520] gitaly[1372]: time="2026-06-30T07:13:06.963Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZPV6R3RWHG56Y76JHGNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-30T07:13:16.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:06.955" grpc.time_ms=7.964 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753vm-test-run-nixbot-gitlab> gitlab # [ 210.999242] setup-gitlab-start[2717]: + git remote add origin http://oauth2:glpat-GVdrDLp71PSu47q7_3EzPm86MQp1OjEH.01.0w1qi4zpu@gitlab/root/test-flake.git1754vm-test-run-nixbot-gitlab> gitlab # [ 211.002069] setup-gitlab-start[2717]: + git push -u origin master1755vm-test-run-nixbot-gitlab> gitlab # [ 211.003371] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccf10e42521097efcc9743cf","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccf10e42521097efcc9743cf: done: 0.018065 sec","job_status":"done","queue_duration_s":0.805162,"scheduling_latency_s":0.805183,"redis_calls":13,"redis_duration_s":0.003445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002396,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018065,"completed_at":"2026-06-30T07:13:04.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1756vm-test-run-nixbot-gitlab> gitlab # [ 211.079235] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33f257da4250ac322d161317","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33f257da4250ac322d161317: start","job_status":"start","queue_duration_s":0.824747,"scheduling_latency_s":0.824777}1757vm-test-run-nixbot-gitlab> gitlab # [ 211.086204] gitaly[1372]: time="2026-06-30T07:13:07.596Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRD88K2S4Q87GXT3MCYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:07.587" grpc.time_ms=9.562 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1758vm-test-run-nixbot-gitlab> gitlab # [ 211.156528] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33f257da4250ac322d161317","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33f257da4250ac322d161317: done: 0.016396 sec","job_status":"done","queue_duration_s":0.824747,"scheduling_latency_s":0.824777,"redis_calls":13,"redis_duration_s":0.003137,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001025,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016396,"completed_at":"2026-06-30T07:13:04.210Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1759vm-test-run-nixbot-gitlab> gitlab # [ 211.234222] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eda7fc56b2b6c3b35732e8d1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eda7fc56b2b6c3b35732e8d1: start","job_status":"start","queue_duration_s":0.842511,"scheduling_latency_s":0.842534}1760vm-test-run-nixbot-gitlab> gitlab # [ 211.235939] gitaly[1372]: time="2026-06-30T07:13:07.998Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-30T07:13:17.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:07.990" grpc.time_ms=7.722 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1761vm-test-run-nixbot-gitlab> gitlab # [ 211.237135] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eda7fc56b2b6c3b35732e8d1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eda7fc56b2b6c3b35732e8d1: done: 0.022253 sec","job_status":"done","queue_duration_s":0.842511,"scheduling_latency_s":0.842534,"redis_calls":13,"redis_duration_s":0.0045190000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022253,"completed_at":"2026-06-30T07:13:04.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1762vm-test-run-nixbot-gitlab> gitlab # [ 211.310452] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3214c8397f8c6992c883fe2","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3214c8397f8c6992c883fe2: start","job_status":"start","queue_duration_s":0.866185,"scheduling_latency_s":0.86621}1763vm-test-run-nixbot-gitlab> gitlab # [ 211.311791] gitaly[1372]: time="2026-06-30T07:13:08.038Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=326 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-30T07:13:38.029" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-30T07:13:08.029" grpc.time_ms=8.921 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1764vm-test-run-nixbot-gitlab> gitlab # [ 211.317686] gitaly[1372]: time="2026-06-30T07:13:08.059Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 duration_ms=159ns method=POST pid=1372 status=200 url="http://unix/api/v4/internal/allowed"1765vm-test-run-nixbot-gitlab> gitlab # [ 211.322792] gitaly[1372]: time="2026-06-30T07:13:08.096Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 duration_ms=36ns method=POST pid=1372 status=200 url="http://unix/api/v4/internal/pre_receive"1766vm-test-run-nixbot-gitlab> gitlab # [ 211.327251] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3214c8397f8c6992c883fe2","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3214c8397f8c6992c883fe2: done: 0.015131 sec","job_status":"done","queue_duration_s":0.866185,"scheduling_latency_s":0.86621,"redis_calls":13,"redis_duration_s":0.002885,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015131,"completed_at":"2026-06-30T07:13:04.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1767vm-test-run-nixbot-gitlab> gitlab # [ 211.398125] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"885d2d85a398b6785e729614","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-885d2d85a398b6785e729614: start","job_status":"start","queue_duration_s":0.885055,"scheduling_latency_s":0.885078}1768vm-test-run-nixbot-gitlab> gitlab # [ 211.473408] gitaly[1372]: time="2026-06-30T07:13:08.097Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:07.899" grpc.time_ms=197.9 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1769vm-test-run-nixbot-gitlab> gitlab # [ 211.474889] gitaly[1372]: time="2026-06-30T07:13:08.127Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:08.126" grpc.time_ms=0.585 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1770vm-test-run-nixbot-gitlab> gitlab # [ 211.476702] setup-gitlab-start[3068]: To http://gitlab/root/test-flake.git1771vm-test-run-nixbot-gitlab> gitlab # [ 211.478254] setup-gitlab-start[3068]: * [new branch] master -> master1772vm-test-run-nixbot-gitlab> gitlab # [ 211.479056] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"885d2d85a398b6785e729614","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-885d2d85a398b6785e729614: done: 0.019332 sec","job_status":"done","queue_duration_s":0.885055,"scheduling_latency_s":0.885078,"redis_calls":13,"redis_duration_s":0.006555,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002411,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004144,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019332,"completed_at":"2026-06-30T07:13:04.274Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1773vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 212.48 seconds)1774vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1775vm-test-run-nixbot-gitlab> gitlab # [ 211.586735] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3519d2f2b48d78e77d98616a","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3519d2f2b48d78e77d98616a: start","job_status":"start","queue_duration_s":0.90568,"scheduling_latency_s":0.905702}1776vm-test-run-nixbot-gitlab> gitlab # [ 211.606126] gitlab-workhorse[1280]: time="2026-06-30T07:13:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521777vm-test-run-nixbot-gitlab> gitlab # [ 211.606604] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:08 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 7921778vm-test-run-nixbot-gitlab> gitlab # [ 211.607770] gitaly[1372]: time="2026-06-30T07:13:08.159Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:08.156" grpc.time_ms=2.364 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1779vm-test-run-nixbot-gitlab> gitlab # [ 211.608927] gitaly[1372]: time="2026-06-30T07:13:08.189Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:08.188" grpc.time_ms=0.906 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1780vm-test-run-nixbot-gitlab> gitlab # [ 211.609949] setup-gitlab-start[3068]: branch 'master' set up to track 'origin/master'.1781vm-test-run-nixbot-gitlab> gitlab # [ 211.611297] systemd[1]: Finished setup-gitlab.service.1782vm-test-run-nixbot-gitlab> gitlab # [ 211.612241] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3519d2f2b48d78e77d98616a","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3519d2f2b48d78e77d98616a: done: 0.014948 sec","job_status":"done","queue_duration_s":0.90568,"scheduling_latency_s":0.905702,"redis_calls":13,"redis_duration_s":0.002816,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014948,"completed_at":"2026-06-30T07:13:04.290Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1783vm-test-run-nixbot-gitlab> gitlab # [ 211.685621] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2db1d29f5857b5f99da2f8ee","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2db1d29f5857b5f99da2f8ee: start","job_status":"start","queue_duration_s":0.921947,"scheduling_latency_s":0.921969}1784vm-test-run-nixbot-gitlab> gitlab # [ 211.687274] gitaly[1372]: time="2026-06-30T07:13:08.319Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=12 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-30T07:13:18.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:08.302" grpc.time_ms=16.576 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1785vm-test-run-nixbot-gitlab> gitlab # [ 211.688235] systemd[1]: Starting nixbot CI service...1786vm-test-run-nixbot-gitlab> gitlab # [ 211.689075] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2db1d29f5857b5f99da2f8ee","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2db1d29f5857b5f99da2f8ee: done: 0.014856 sec","job_status":"done","queue_duration_s":0.921947,"scheduling_latency_s":0.921969,"redis_calls":13,"redis_duration_s":0.002942,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014856,"completed_at":"2026-06-30T07:13:04.306Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1787vm-test-run-nixbot-gitlab> gitlab # [ 211.742339] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b4128b474d9b006aa77e27a","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b4128b474d9b006aa77e27a: start","job_status":"start","queue_duration_s":0.938226,"scheduling_latency_s":0.938248}1788vm-test-run-nixbot-gitlab> gitlab # [ 211.816677] gitaly[1372]: time="2026-06-30T07:13:08.339Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-30T07:13:18.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:08.324" grpc.time_ms=15.315 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789vm-test-run-nixbot-gitlab> gitlab # [ 211.817606] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b4128b474d9b006aa77e27a","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b4128b474d9b006aa77e27a: done: 0.015223 sec","job_status":"done","queue_duration_s":0.938226,"scheduling_latency_s":0.938248,"redis_calls":13,"redis_duration_s":0.0031739999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002254,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015223,"completed_at":"2026-06-30T07:13:04.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1790vm-test-run-nixbot-gitlab> gitlab # [ 211.844930] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecdb4516dace47af98847c3e","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecdb4516dace47af98847c3e: start","job_status":"start","queue_duration_s":0.95474,"scheduling_latency_s":0.954763}1791vm-test-run-nixbot-gitlab> gitlab # [ 211.915975] gitaly[1372]: time="2026-06-30T07:13:08.366Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1555 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=28 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-30T07:13:18.027" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:08.327" grpc.time_ms=38.564 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1792vm-test-run-nixbot-gitlab> gitlab # [ 211.917756] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecdb4516dace47af98847c3e","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecdb4516dace47af98847c3e: done: 0.020121 sec","job_status":"done","queue_duration_s":0.95474,"scheduling_latency_s":0.954763,"redis_calls":13,"redis_duration_s":0.007834,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005789,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020121,"completed_at":"2026-06-30T07:13:04.344Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1793vm-test-run-nixbot-gitlab> gitlab # [ 211.995316] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf02216b44a871d64fc07b27","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf02216b44a871d64fc07b27: start","job_status":"start","queue_duration_s":0.976211,"scheduling_latency_s":0.976234}1794vm-test-run-nixbot-gitlab> gitlab # [ 211.996621] gitaly[1372]: time="2026-06-30T07:13:08.369Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:18.037" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:08.357" grpc.time_ms=11.233 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1795vm-test-run-nixbot-gitlab> gitlab # [ 211.997484] gitaly[1372]: time="2026-06-30T07:13:08.383Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 duration_ms=166ns method=POST pid=1372 status=200 url="http://unix/api/v4/internal/post_receive"1796vm-test-run-nixbot-gitlab> gitlab # [ 211.998349] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf02216b44a871d64fc07b27","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf02216b44a871d64fc07b27: done: 0.014922 sec","job_status":"done","queue_duration_s":0.976211,"scheduling_latency_s":0.976234,"redis_calls":13,"redis_duration_s":0.002971,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014922,"completed_at":"2026-06-30T07:13:04.360Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1797vm-test-run-nixbot-gitlab> gitlab # [ 212.091091] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"242bd5ad1c9521959d46b136","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-242bd5ad1c9521959d46b136: start","job_status":"start","queue_duration_s":0.992403,"scheduling_latency_s":0.992424}1798vm-test-run-nixbot-gitlab> gitlab # [ 212.093058] gitaly[1372]: time="2026-06-30T07:13:08.385Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:08.216" grpc.time_ms=168.601 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1799vm-test-run-nixbot-gitlab> gitlab # [ 212.094308] gitaly[1372]: time="2026-06-30T07:13:08.399Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 diskcache=7bd0fa9b-b2a8-47ab-9289-7f8d28352ede grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:07.842" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1800vm-test-run-nixbot-gitlab> gitlab # [ 212.106387] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"242bd5ad1c9521959d46b136","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-242bd5ad1c9521959d46b136: done: 0.014768 sec","job_status":"done","queue_duration_s":0.992403,"scheduling_latency_s":0.992424,"redis_calls":13,"redis_duration_s":0.002961,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014768,"completed_at":"2026-06-30T07:13:04.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1801vm-test-run-nixbot-gitlab> gitlab # [ 212.178225] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"201f043b33a9090688e860bd","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-201f043b33a9090688e860bd: start","job_status":"start","queue_duration_s":1.008451,"scheduling_latency_s":1.008474}1802vm-test-run-nixbot-gitlab> gitlab # [ 212.248241] gitaly[1372]: time="2026-06-30T07:13:08.399Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=172 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13445 command.oublock=32 command.real_time_ms=547 command.spawn_token_fork_ms=2 command.system_time_ms=112 command.user_time_ms=60 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:07.842" grpc.time_ms=556.343 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1803vm-test-run-nixbot-gitlab> gitlab # [ 212.250174] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"201f043b33a9090688e860bd","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-201f043b33a9090688e860bd: done: 0.014915 sec","job_status":"done","queue_duration_s":1.008451,"scheduling_latency_s":1.008474,"redis_calls":13,"redis_duration_s":0.002938,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014915,"completed_at":"2026-06-30T07:13:04.393Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1804vm-test-run-nixbot-gitlab> gitlab # [ 212.335259] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c0af721d827c1f3b3004b0c","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c0af721d827c1f3b3004b0c: start","job_status":"start","queue_duration_s":1.024632,"scheduling_latency_s":1.024654}1805vm-test-run-nixbot-gitlab> gitlab # [ 212.335849] gitaly[1372]: time="2026-06-30T07:13:08.430Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=13 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-30T07:13:18.088" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:08.408" grpc.time_ms=21.104 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806vm-test-run-nixbot-gitlab> gitlab # [ 212.336929] gitaly[1372]: time="2026-06-30T07:13:08.475Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:08.473" grpc.time_ms=1.295 pid=1372 span.kind=server system=grpc1807vm-test-run-nixbot-gitlab> gitlab # [ 212.338152] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c0af721d827c1f3b3004b0c","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c0af721d827c1f3b3004b0c: done: 0.014923 sec","job_status":"done","queue_duration_s":1.024632,"scheduling_latency_s":1.024654,"redis_calls":13,"redis_duration_s":0.002872,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014923,"completed_at":"2026-06-30T07:13:04.409Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1808vm-test-run-nixbot-gitlab> gitlab # [ 212.409500] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ea1a6ed664cfae6f2c89077","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ea1a6ed664cfae6f2c89077: start","job_status":"start","queue_duration_s":1.040963,"scheduling_latency_s":1.040985}1809vm-test-run-nixbot-gitlab> gitlab # [ 212.410782] gitaly[1372]: time="2026-06-30T07:13:08.501Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:08.501" grpc.time_ms=0.343 pid=1372 span.kind=server system=grpc1810vm-test-run-nixbot-gitlab> gitlab # [ 212.423049] gitaly[1372]: time="2026-06-30T07:13:08.520Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 diskcache=a55e6152-8b8b-4104-8b1f-a7dcf91ed829 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-30T07:13:18.125" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:08.446" pid=1372 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1811vm-test-run-nixbot-gitlab> gitlab # [ 212.439914] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ea1a6ed664cfae6f2c89077","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ea1a6ed664cfae6f2c89077: done: 0.026087 sec","job_status":"done","queue_duration_s":1.040963,"scheduling_latency_s":1.040985,"redis_calls":13,"redis_duration_s":0.010057,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002434,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007623,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026087,"completed_at":"2026-06-30T07:13:04.436Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1812vm-test-run-nixbot-gitlab> gitlab # [ 212.531382] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c7be431f7500673b47612a1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c7be431f7500673b47612a1: start","job_status":"start","queue_duration_s":1.070213,"scheduling_latency_s":1.070237}1813vm-test-run-nixbot-gitlab> gitlab # [ 212.532758] gitaly[1372]: time="2026-06-30T07:13:08.520Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4962 command.oublock=8 command.real_time_ms=58 command.spawn_token_fork_ms=0 command.system_time_ms=35 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-30T07:13:18.125" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:08.446" grpc.time_ms=74.244 pid=1372 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1814vm-test-run-nixbot-gitlab> gitlab # [ 212.605356] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:04.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c7be431f7500673b47612a1","created_at":"2026-06-30T07:13:03.287Z","trace_propagation_headers":{"sentry-trace":"9c0f12e89d064bce8a9d4c32925603ec-8064a08ad7424aef","baggage":"sentry-trace_id=9c0f12e89d064bce8a9d4c32925603ec,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"743d47487995e1fb0e39c9e4e1cc7939","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:03.369Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c7be431f7500673b47612a1: done: 0.022828 sec","job_status":"done","queue_duration_s":1.070213,"scheduling_latency_s":1.070237,"redis_calls":13,"redis_duration_s":0.010112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003293,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006819,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022828,"completed_at":"2026-06-30T07:13:04.462Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1815vm-test-run-nixbot-gitlab> gitlab # [ 212.607412] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:05.871Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"ed73f8f2e34ffa299c5dc3c3","created_at":"2026-06-30T07:13:05.843Z","trace_propagation_headers":{"sentry-trace":"64944dfd78c14d589f9ca35f57ee2c95-a6e0f068202a4593","baggage":"sentry-trace_id=64944dfd78c14d589f9ca35f57ee2c95,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPK5TA2ZS6H59TPJA4NJ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:509e1afd55f724a00464acb0a7c061b30c3ffaaa6ab52ef90c1b5346093cbd52","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:05.865Z","job_size_bytes":451,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-ed73f8f2e34ffa299c5dc3c3: start","job_status":"start","queue_duration_s":0.005338,"scheduling_latency_s":0.005382}1816vm-test-run-nixbot-gitlab> gitlab # [ 212.679196] gitaly[1372]: time="2026-06-30T07:13:08.540Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-30T07:13:18.218" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:08.538" grpc.time_ms=0.179 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817vm-test-run-nixbot-gitlab> gitlab # [ 212.683251] gitaly[1372]: time="2026-06-30T07:13:08.557Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=943 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-30T07:13:18.227" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:08.548" grpc.time_ms=8.874 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818vm-test-run-nixbot-gitlab> gitlab # [ 212.684195] gitaly[1372]: time="2026-06-30T07:13:08.633Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-30T07:13:38.028" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:08.629" grpc.time_ms=4.577 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1819vm-test-run-nixbot-gitlab> gitlab # [ 212.691843] gitaly[1372]: time="2026-06-30T07:13:08.655Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-30T07:13:18.004" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:08.654" grpc.time_ms=0.192 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1820vm-test-run-nixbot-gitlab> gitlab # [ 212.762476] sidekiq-cluster[2141]: {"severity":"WARN","time":"2026-06-30T07:13:05.964Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"ed73f8f2e34ffa299c5dc3c3","created_at":"2026-06-30T07:13:05.843Z","trace_propagation_headers":{"sentry-trace":"64944dfd78c14d589f9ca35f57ee2c95-a6e0f068202a4593","baggage":"sentry-trace_id=64944dfd78c14d589f9ca35f57ee2c95,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPK5TA2ZS6H59TPJA4NJ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:509e1afd55f724a00464acb0a7c061b30c3ffaaa6ab52ef90c1b5346093cbd52","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:05.865Z","job_size_bytes":451,"pid":2141,"sidekiq_tid":"lys9","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-ed73f8f2e34ffa299c5dc3c3: fail: 0.093154 sec","job_status":"fail","queue_duration_s":0.005338,"scheduling_latency_s":0.005382,"redis_calls":11,"redis_duration_s":0.003552,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000726,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001998,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000441676,"duration_s":0.093154,"completed_at":"2026-06-30T07:13:05.963Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.016764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1821vm-test-run-nixbot-gitlab> gitlab # [ 213.339531] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:06.643Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"49641bbe40352878fc734a6e","created_at":"2026-06-30T07:13:06.634Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.641Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-49641bbe40352878fc734a6e: start","job_status":"start","queue_duration_s":0.00166,"scheduling_latency_s":0.001704}1822vm-test-run-nixbot-gitlab> gitlab # [ 213.340405] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:06.699Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"49641bbe40352878fc734a6e","created_at":"2026-06-30T07:13:06.634Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.641Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-49641bbe40352878fc734a6e: done: 0.05629 sec","job_status":"done","queue_duration_s":0.00166,"scheduling_latency_s":0.001704,"redis_calls":13,"redis_duration_s":0.003995,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001099,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001732,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001164,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.028106924,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.05629,"completed_at":"2026-06-30T07:13:06.699Z","load_balancing_strategy":"primary","db_duration_s":0.008525,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1823vm-test-run-nixbot-gitlab> gitlab # [ 213.435401] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:06.701Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"9ebee41e2ece66cd747f4748","created_at":"2026-06-30T07:13:06.651Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.657Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-9ebee41e2ece66cd747f4748: start","job_status":"start","queue_duration_s":0.043489,"scheduling_latency_s":0.04352}1824vm-test-run-nixbot-gitlab> gitlab # [ 213.436917] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:06.790Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"9ebee41e2ece66cd747f4748","created_at":"2026-06-30T07:13:06.651Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.657Z","job_size_bytes":2,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-9ebee41e2ece66cd747f4748: done: 0.08896 sec","job_status":"done","queue_duration_s":0.043489,"scheduling_latency_s":0.04352,"redis_calls":14,"redis_duration_s":0.007151,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.002318,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002221,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002612,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.060074395,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.08896,"completed_at":"2026-06-30T07:13:06.789Z","load_balancing_strategy":"primary","db_duration_s":0.018195,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1825vm-test-run-nixbot-gitlab> gitlab # [ 213.533297] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:06.904Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"a75188b1380eb25ae17c8c8e","created_at":"2026-06-30T07:13:06.897Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.903Z","job_size_bytes":5,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-a75188b1380eb25ae17c8c8e: start","job_status":"start","queue_duration_s":0.001785,"scheduling_latency_s":0.001833}1826vm-test-run-nixbot-gitlab> gitlab # [ 213.536054] gitaly[1372]: time="2026-06-30T07:13:08.661Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-30T07:13:18.006" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-30T07:13:08.657" grpc.time_ms=4.342 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1827vm-test-run-nixbot-gitlab> gitlab # [ 213.621652] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:06.999Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"a75188b1380eb25ae17c8c8e","created_at":"2026-06-30T07:13:06.897Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.903Z","job_size_bytes":5,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-a75188b1380eb25ae17c8c8e: done: 0.094832 sec","job_status":"done","queue_duration_s":0.001785,"scheduling_latency_s":0.001833,"redis_calls":12,"redis_duration_s":0.0035210000000000003,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00077,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002027,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000724,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000394184,"duration_s":0.094832,"completed_at":"2026-06-30T07:13:06.999Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017174,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1828vm-test-run-nixbot-gitlab> gitlab # [ 213.711356] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:07.001Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f3e223e27a8bf9b8f62b89f7","created_at":"2026-06-30T07:13:06.927Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.933Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f3e223e27a8bf9b8f62b89f7: start","job_status":"start","queue_duration_s":0.06759,"scheduling_latency_s":0.067614}1829vm-test-run-nixbot-gitlab> gitlab # [ 213.713972] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:07.204Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f3e223e27a8bf9b8f62b89f7","created_at":"2026-06-30T07:13:06.927Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.933Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f3e223e27a8bf9b8f62b89f7: done: 0.202787 sec","job_status":"done","queue_duration_s":0.06759,"scheduling_latency_s":0.067614,"redis_calls":11,"redis_duration_s":0.003024,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000849,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.131,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.131,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.133858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384686,"duration_s":0.202787,"completed_at":"2026-06-30T07:13:07.204Z","load_balancing_strategy":"primary","db_duration_s":0.069498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1830vm-test-run-nixbot-gitlab> gitlab # [ 213.794436] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:07.206Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"593315b34177d6cf0dd990d0","created_at":"2026-06-30T07:13:06.936Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.941Z","job_size_bytes":89,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-593315b34177d6cf0dd990d0: start","job_status":"start","queue_duration_s":0.264923,"scheduling_latency_s":0.264946}1831vm-test-run-nixbot-gitlab> gitlab # [ 213.795875] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:07.266Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"593315b34177d6cf0dd990d0","created_at":"2026-06-30T07:13:06.936Z","trace_propagation_headers":{"sentry-trace":"350f245b05f04d54a22eab0c4f473414-6f3334c0a6394ef4","baggage":"sentry-trace_id=350f245b05f04d54a22eab0c4f473414,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPV6R3RWHG56Y76JHGNG","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:06.941Z","job_size_bytes":89,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-593315b34177d6cf0dd990d0: done: 0.059916 sec","job_status":"done","queue_duration_s":0.264923,"scheduling_latency_s":0.264946,"redis_calls":11,"redis_duration_s":0.003975,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000686,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000729,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000371276,"duration_s":0.059916,"completed_at":"2026-06-30T07:13:07.266Z","load_balancing_strategy":"primary","db_duration_s":0.015846,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1832vm-test-run-nixbot-gitlab> gitlab # [ 213.873113] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:08.278Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 e6cf5e803699fa6869c51ec2f1ff941148940ff0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"405363f448fd128841a785d8","created_at":"2026-06-30T07:13:08.252Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-37b53ec6a02547d3","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:08.275Z","job_size_bytes":185,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-405363f448fd128841a785d8: start","job_status":"start","queue_duration_s":0.002883,"scheduling_latency_s":0.002928}1833vm-test-run-nixbot-gitlab> gitlab # [ 213.944860] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:08.297Z","message":"changes: \"0000000000000000000000000000000000000000 e6cf5e803699fa6869c51ec2f1ff941148940ff0 refs/heads/master\\n\""}1834vm-test-run-nixbot-gitlab> gitlab # [ 213.945960] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:09.075Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 e6cf5e803699fa6869c51ec2f1ff941148940ff0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"405363f448fd128841a785d8","created_at":"2026-06-30T07:13:08.252Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-37b53ec6a02547d3","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:08.275Z","job_size_bytes":185,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-405363f448fd128841a785d8: done: 0.7969 sec","job_status":"done","queue_duration_s":0.002883,"scheduling_latency_s":0.002928,"gitaly_calls":13,"gitaly_duration_s":0.226886,"redis_calls":200,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.066414,"redis_read_bytes":941,"redis_write_bytes":19810,"redis_cache_calls":14,"redis_cache_duration_s":0.004841,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":32,"redis_feature_flag_duration_s":0.007655,"redis_feature_flag_read_bytes":540,"redis_feature_flag_write_bytes":4279,"redis_queues_calls":8,"redis_queues_duration_s":0.001331,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003267,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.021656,"redis_repository_cache_read_bytes":233,"redis_repository_cache_write_bytes":6319,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.027664,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":51,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":51,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":51,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.233,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.261,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.441039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000771047,"exclusive_lock_hold_duration_s":0.027976181,"duration_s":0.7969,"completed_at":"2026-06-30T07:13:09.075Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077005,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1835vm-test-run-nixbot-gitlab> gitlab # [ 214.024866] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:09.077Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"3a7bdf9243810eebf4c76992","created_at":"2026-06-30T07:13:08.375Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-b0550bcf55a741d8","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:08.380Z","job_size_bytes":46,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3a7bdf9243810eebf4c76992: start","job_status":"start","queue_duration_s":0.696536,"scheduling_latency_s":0.69656}1836vm-test-run-nixbot-gitlab> gitlab # [ 214.026157] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:09.155Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"3a7bdf9243810eebf4c76992","created_at":"2026-06-30T07:13:08.375Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-b0550bcf55a741d8","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:08.380Z","job_size_bytes":46,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3a7bdf9243810eebf4c76992: done: 0.078289 sec","job_status":"done","queue_duration_s":0.696536,"scheduling_latency_s":0.69656,"gitaly_calls":2,"gitaly_duration_s":0.013972,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007718999999999999,"redis_read_bytes":68,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001499,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000597,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001651,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003289,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000683,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000357308,"duration_s":0.078289,"completed_at":"2026-06-30T07:13:09.155Z","load_balancing_strategy":"primary","db_duration_s":0.009553,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1837vm-test-run-nixbot-gitlab> gitlab # [ 214.114252] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:09.157Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ba20b6e3f50038fb8188c6b5","created_at":"2026-06-30T07:13:08.396Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-b0550bcf55a741d8","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:08.403Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ba20b6e3f50038fb8188c6b5: start","job_status":"start","queue_duration_s":0.753105,"scheduling_latency_s":0.753128}1838vm-test-run-nixbot-gitlab> gitlab # [ 214.115797] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:09.239Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ba20b6e3f50038fb8188c6b5","created_at":"2026-06-30T07:13:08.396Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-b0550bcf55a741d8","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:08.403Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ba20b6e3f50038fb8188c6b5: done: 0.082297 sec","job_status":"done","queue_duration_s":0.753105,"scheduling_latency_s":0.753128,"gitaly_calls":1,"gitaly_duration_s":0.032321,"redis_calls":15,"redis_duration_s":0.003785,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000705,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.059853975000000004,"duration_s":0.082297,"completed_at":"2026-06-30T07:13:09.239Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1839vm-test-run-nixbot-gitlab> gitlab # [ 214.236045] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:09.241Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"bcb6b60fd382bdbefc24981a","created_at":"2026-06-30T07:13:08.533Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-b0550bcf55a741d8","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:08.535Z","job_size_bytes":23,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-bcb6b60fd382bdbefc24981a: start","job_status":"start","queue_duration_s":0.705407,"scheduling_latency_s":0.70543}1840vm-test-run-nixbot-gitlab> gitlab # [ 214.236884] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:09.398Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"bcb6b60fd382bdbefc24981a","created_at":"2026-06-30T07:13:08.533Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-b0550bcf55a741d8","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:08.535Z","job_size_bytes":23,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-bcb6b60fd382bdbefc24981a: done: 0.157421 sec","job_status":"done","queue_duration_s":0.705407,"scheduling_latency_s":0.70543,"gitaly_calls":1,"gitaly_duration_s":0.011405,"redis_calls":16,"redis_duration_s":0.004489,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000354,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001204,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002195,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.115,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.115,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.157421,"completed_at":"2026-06-30T07:13:09.398Z","load_balancing_strategy":"primary","db_duration_s":0.115271,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1841vm-test-run-nixbot-gitlab> gitlab # [ 214.315670] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:09.400Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"59a878c9fc4b1c9d58dc7b8a","created_at":"2026-06-30T07:13:09.006Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-b0550bcf55a741d8","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b769099ae8e891593691545824f6fe6331ba718708b98b07d743b4dee199f9c1","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:09.012Z","job_size_bytes":467,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-59a878c9fc4b1c9d58dc7b8a: start","job_status":"start","queue_duration_s":0.387409,"scheduling_latency_s":0.387438}1842vm-test-run-nixbot-gitlab> gitlab # [ 214.317211] sidekiq-cluster[2141]: {"severity":"WARN","time":"2026-06-30T07:13:10.311Z","class":"CreatePipelineWorker","correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"e6cf5e803699fa6869c51ec2f1ff941148940ff0","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"e6cf5e803699fa6869c51ec2f1ff941148940ff0","ref":"refs/heads/master"},"retry":0}1843vm-test-run-nixbot-gitlab> gitlab # [ 214.393449] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:10.317Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"59a878c9fc4b1c9d58dc7b8a","created_at":"2026-06-30T07:13:09.006Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-b0550bcf55a741d8","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b769099ae8e891593691545824f6fe6331ba718708b98b07d743b4dee199f9c1","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:09.012Z","job_size_bytes":467,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-59a878c9fc4b1c9d58dc7b8a: done: 0.917213 sec","job_status":"done","queue_duration_s":0.387409,"scheduling_latency_s":0.387438,"gitaly_calls":7,"gitaly_duration_s":0.064994,"redis_calls":53,"redis_duration_s":0.011605,"redis_read_bytes":171,"redis_write_bytes":4426,"redis_cache_calls":4,"redis_cache_duration_s":0.001083,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.00495,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001507,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003009,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.051,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.725496,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000348089,"duration_s":0.917213,"completed_at":"2026-06-30T07:13:10.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.090006,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1844vm-test-run-nixbot-gitlab> gitlab # [ 214.467593] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:10.319Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"78f93e157bdba553acab583b","created_at":"2026-06-30T07:13:09.141Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-65b3885ef5c24588","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:09.145Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-78f93e157bdba553acab583b: start","job_status":"start","queue_duration_s":1.173525,"scheduling_latency_s":1.173548}1845vm-test-run-nixbot-gitlab> gitlab # [ 214.469359] gitaly[1372]: time="2026-06-30T07:13:08.680Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:18.023" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:08.674" grpc.time_ms=5.93 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1846vm-test-run-nixbot-gitlab> gitlab # [ 214.470666] nixbot[3192]: {"ts": "2026-06-30T07:13:11.271Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1847vm-test-run-nixbot-gitlab> gitlab # [ 214.471874] nixbot[3192]: {"ts": "2026-06-30T07:13:11.291Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1848vm-test-run-nixbot-gitlab> gitlab # [ 214.473253] nixbot[3192]: {"ts": "2026-06-30T07:13:11.304Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1849vm-test-run-nixbot-gitlab> gitlab # [ 214.474131] nixbot[3192]: {"ts": "2026-06-30T07:13:11.308Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1850vm-test-run-nixbot-gitlab> gitlab # [ 214.475183] nixbot[3192]: {"ts": "2026-06-30T07:13:11.311Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1851vm-test-run-nixbot-gitlab> gitlab # [ 214.476165] nixbot[3192]: {"ts": "2026-06-30T07:13:11.315Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1852vm-test-run-nixbot-gitlab> gitlab # [ 214.476594] nixbot[3192]: {"ts": "2026-06-30T07:13:11.318Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1853vm-test-run-nixbot-gitlab> gitlab # [ 214.548765] nixbot[3192]: {"ts": "2026-06-30T07:13:11.321Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1854vm-test-run-nixbot-gitlab> gitlab # [ 214.550103] nixbot[3192]: {"ts": "2026-06-30T07:13:11.326Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1855vm-test-run-nixbot-gitlab> gitlab # [ 214.550935] nixbot[3192]: {"ts": "2026-06-30T07:13:11.329Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1856vm-test-run-nixbot-gitlab> gitlab # [ 214.552210] nixbot[3192]: {"ts": "2026-06-30T07:13:11.334Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1857vm-test-run-nixbot-gitlab> gitlab # [ 214.553060] nixbot[3192]: {"ts": "2026-06-30T07:13:11.338Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1858vm-test-run-nixbot-gitlab> gitlab # [ 214.553992] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:10.362Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"78f93e157bdba553acab583b","created_at":"2026-06-30T07:13:09.141Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-65b3885ef5c24588","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:09.145Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-78f93e157bdba553acab583b: done: 0.043913 sec","job_status":"done","queue_duration_s":1.173525,"scheduling_latency_s":1.173548,"redis_calls":19,"redis_duration_s":0.0046,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001034,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000676,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00212,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00077,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.010658059000000001,"duration_s":0.043913,"completed_at":"2026-06-30T07:13:10.362Z","load_balancing_strategy":"primary","db_duration_s":0.006249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1859vm-test-run-nixbot-gitlab> gitlab # [ 214.629968] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:10.364Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"d927456128ff9aebccf07253","created_at":"2026-06-30T07:13:10.345Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-a8343802f3934876","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:10.352Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-d927456128ff9aebccf07253: start","job_status":"start","queue_duration_s":0.01165,"scheduling_latency_s":0.011673}1860vm-test-run-nixbot-gitlab> gitlab # [ 214.631227] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:10.441Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"d927456128ff9aebccf07253","created_at":"2026-06-30T07:13:10.345Z","trace_propagation_headers":{"sentry-trace":"86965ef579c642729945a724ce4a5cf1-a8343802f3934876","baggage":"sentry-trace_id=86965ef579c642729945a724ce4a5cf1,sentry-release=Unknown"},"correlation_id":"01KWBNZRGQKQGP0ZZ52AZMHMW1","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:10.352Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-d927456128ff9aebccf07253: done: 0.077434 sec","job_status":"done","queue_duration_s":0.01165,"scheduling_latency_s":0.011673,"redis_calls":11,"redis_duration_s":0.00286,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001885,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000485,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000399213,"duration_s":0.077434,"completed_at":"2026-06-30T07:13:10.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018664,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1861vm-test-run-nixbot-gitlab> gitlab # [ 214.701864] gitaly[1372]: time="2026-06-30T07:13:08.692Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:18.033" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:08.685" grpc.time_ms=6.15 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862vm-test-run-nixbot-gitlab> gitlab # [ 214.703130] nixbot[3192]: {"ts": "2026-06-30T07:13:11.342Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1863vm-test-run-nixbot-gitlab> gitlab # [ 214.703898] nixbot[3192]: {"ts": "2026-06-30T07:13:11.344Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1864vm-test-run-nixbot-gitlab> gitlab # [ 214.705173] nixbot[3192]: {"ts": "2026-06-30T07:13:11.347Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1865vm-test-run-nixbot-gitlab> gitlab # [ 214.705906] nixbot[3192]: {"ts": "2026-06-30T07:13:11.350Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1866vm-test-run-nixbot-gitlab> gitlab # [ 214.707113] nixbot[3192]: {"ts": "2026-06-30T07:13:11.352Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1867vm-test-run-nixbot-gitlab> gitlab # [ 214.707956] nixbot[3192]: {"ts": "2026-06-30T07:13:11.354Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1868vm-test-run-nixbot-gitlab> gitlab # [ 214.709136] nixbot[3192]: {"ts": "2026-06-30T07:13:11.358Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1869vm-test-run-nixbot-gitlab> gitlab # [ 214.711694] gitaly[1372]: time="2026-06-30T07:13:08.727Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=969 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-30T07:13:38.117" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:08.717" grpc.time_ms=9.867 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1870vm-test-run-nixbot-gitlab> gitlab # [ 214.790066] gitaly[1372]: time="2026-06-30T07:13:09.046Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-30T07:13:19.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:09.044" grpc.time_ms=1.488 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1871vm-test-run-nixbot-gitlab> gitlab # [ 214.791263] gitaly[1372]: time="2026-06-30T07:13:09.118Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-30T13:13:09.114" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:09.114" grpc.time_ms=2.547 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1872vm-test-run-nixbot-gitlab> gitlab # [ 214.792103] gitaly[1372]: time="2026-06-30T07:13:09.131Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-30T07:13:39.023" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:09.124" grpc.time_ms=6.653 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873vm-test-run-nixbot-gitlab> gitlab # [ 214.797369] gitaly[1372]: time="2026-06-30T07:13:09.208Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 diskcache=37864363-ca5d-40d4-ab2b-a8cffd7cde59 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T13:13:09.178" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:09.179" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874vm-test-run-nixbot-gitlab> gitlab # [ 214.813517] gitaly[1372]: time="2026-06-30T07:13:09.209Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1277 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T13:13:09.178" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:09.179" grpc.time_ms=29.462 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1875vm-test-run-nixbot-gitlab> gitlab # [ 214.893264] gitaly[1372]: time="2026-06-30T07:13:09.389Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-30T07:13:39.279" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:09.380" grpc.time_ms=9.036 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876vm-test-run-nixbot-gitlab> gitlab # [ 214.894812] gitaly[1372]: time="2026-06-30T07:13:09.438Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:39.336" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:09.437" grpc.time_ms=0.611 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877vm-test-run-nixbot-gitlab> gitlab # [ 214.896046] gitaly[1372]: time="2026-06-30T07:13:09.451Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-30T07:13:19.013" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:09.443" grpc.time_ms=7.81 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878vm-test-run-nixbot-gitlab> gitlab # [ 214.899327] gitaly[1372]: time="2026-06-30T07:13:09.554Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-30T07:13:39.452" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:09.552" grpc.time_ms=1.049 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879vm-test-run-nixbot-gitlab> gitlab # [ 214.976546] gitaly[1372]: time="2026-06-30T07:13:09.563Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=160 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-30T07:13:39.458" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:09.559" grpc.time_ms=3.481 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880vm-test-run-nixbot-gitlab> gitlab # [ 214.978118] nixbot[3192]: INFO: Started server process [3192]1881vm-test-run-nixbot-gitlab> gitlab # [ 214.978907] nixbot[3192]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1882vm-test-run-nixbot-gitlab> gitlab # [ 214.980097] gitaly[1372]: time="2026-06-30T07:13:09.650Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:19.213" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:09.643" grpc.time_ms=6.686 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1883vm-test-run-nixbot-gitlab> gitlab # [ 214.980988] gitaly[1372]: time="2026-06-30T07:13:10.257Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=969 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-30T07:13:40.144" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:10.245" grpc.time_ms=11.92 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884vm-test-run-nixbot-gitlab> gitlab # [ 214.993403] gitaly[1372]: time="2026-06-30T07:13:10.271Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZRGQKQGP0ZZ52AZMHMW1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-30T07:13:40.159" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:10.259" grpc.time_ms=11.209 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885vm-test-run-nixbot-gitlab> gitlab # [ 215.097474] gitaly[1372]: time="2026-06-30T07:13:12.029Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZWNDM4QCX3MM8DBF9KCG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-30T07:13:42.019" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:12.019" grpc.time_ms=10.34 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1886vm-test-run-nixbot-gitlab> gitlab # [ 215.114587] nixbot[3192]: INFO: ::ffff:127.0.0.1:58658 - "GET /health HTTP/1.1" 200 OK1887vm-test-run-nixbot-gitlab> gitlab # [ 215.125863] systemd[1]: Started nixbot CI service.1888vm-test-run-nixbot-gitlab> gitlab # [ 215.126928] systemd[1]: Reached target Multi-User System.1889vm-test-run-nixbot-gitlab> gitlab # [ 215.128562] systemd[1]: Startup finished in 1.514s (kernel) + 4.358s (initrd) + 3min 29.255s (userspace) = 3min 35.128s.1890vm-test-run-nixbot-gitlab> gitlab # [ 215.132048] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:12 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 2101891vm-test-run-nixbot-gitlab> gitlab # [ 215.132828] nixbot[3192]: {"ts": "2026-06-30T07:13:12.065Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1892vm-test-run-nixbot-gitlab> gitlab # [ 215.147381] nixbot[3192]: {"ts": "2026-06-30T07:13:12.079Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1893vm-test-run-nixbot-gitlab> gitlab # [ 215.226137] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 681894vm-test-run-nixbot-gitlab> gitlab # [ 215.235879] nixbot[3192]: {"ts": "2026-06-30T07:13:12.168Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1895vm-test-run-nixbot-gitlab> gitlab # [ 215.237840] nixbot[3192]: {"ts": "2026-06-30T07:13:12.169Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1896vm-test-run-nixbot-gitlab> gitlab # [ 215.399599] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1591897vm-test-run-nixbot-gitlab> gitlab # [ 215.401343] nixbot[3192]: {"ts": "2026-06-30T07:13:12.332Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1898vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 4.17 seconds)1899vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1900vm-test-run-nixbot-gitlab> gitlab # [ 215.873676] nixbot[3192]: INFO: ::ffff:127.0.0.1:58674 - "GET /health HTTP/1.1" 200 OK1901vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.05 seconds)1902vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1903vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1904vm-test-run-nixbot-gitlab> gitlab # [ 216.028406] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 1071905vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds)1906vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-30T07:13:12.313Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1907vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 0.16 seconds)1908vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1909vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1910vm-test-run-nixbot-gitlab> gitlab # [ 216.114955] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 291911vm-test-run-nixbot-gitlab> gitlab # [ 216.117712] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 6871912vm-test-run-nixbot-gitlab> gitlab # [ 216.118972] nixbot[3192]: {"ts": "2026-06-30T07:13:13.050Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1913vm-test-run-nixbot-gitlab> gitlab # [ 216.169728] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 411914vm-test-run-nixbot-gitlab> gitlab # [ 216.170565] nixbot[3192]: {"ts": "2026-06-30T07:13:13.103Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1915vm-test-run-nixbot-gitlab> gitlab # [ 216.209848] gitaly[1372]: time="2026-06-30T07:13:13.142Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXV3A76DG0PH8K4NHCJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:13.134" grpc.time_ms=7.153 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1916vm-test-run-nixbot-gitlab> gitlab # [ 216.213187] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.54.0" 841917vm-test-run-nixbot-gitlab> gitlab # [ 216.240808] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 671918vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1919vm-test-run-nixbot-gitlab> gitlab # [ 216.241949] nixbot[3192]: {"ts": "2026-06-30T07:13:13.174Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1920vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1921vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1922vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1923vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 272 bytes | 136.00 KiB/s, done.1924vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1925vm-test-run-nixbot-gitlab> gitlab # [ 216.306320] gitaly[1372]: time="2026-06-30T07:13:13.238Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZXYV1ZBT0PQVV6TGFSW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-30T07:13:43.229" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:13.230" grpc.time_ms=7.962 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1926vm-test-run-nixbot-gitlab> gitlab # [ 216.346104] gitaly[1372]: time="2026-06-30T07:13:13.278Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZXYV1ZBT0PQVV6TGFSW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:43.277" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:13.277" grpc.time_ms=0.463 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1927vm-test-run-nixbot-gitlab> gitlab # [ 216.380181] gitaly[1372]: time="2026-06-30T07:13:13.312Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZXYV1ZBT0PQVV6TGFSW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-30T07:13:23.002" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:13.303" grpc.time_ms=8.768 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928vm-test-run-nixbot-gitlab> gitlab # [ 216.443398] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1951929vm-test-run-nixbot-gitlab> gitlab # [ 216.450139] nixbot[3192]: {"ts": "2026-06-30T07:13:13.380Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1930vm-test-run-nixbot-gitlab> gitlab # [ 216.478518] gitaly[1372]: time="2026-06-30T07:13:13.410Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-30T07:13:23.001" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:13.402" grpc.time_ms=8.156 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931vm-test-run-nixbot-gitlab> gitlab # [ 216.530410] gitaly[1372]: time="2026-06-30T07:13:13.462Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-30T07:13:43.452" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-30T07:13:13.453" grpc.time_ms=9.654 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932vm-test-run-nixbot-gitlab> gitlab # [ 216.567540] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1121933vm-test-run-nixbot-gitlab> gitlab # [ 216.571932] nixbot[3192]: {"ts": "2026-06-30T07:13:13.500Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1934vm-test-run-nixbot-gitlab> gitlab # [ 216.576495] nixbot[3192]: {"ts": "2026-06-30T07:13:13.505Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1935vm-test-run-nixbot-gitlab> gitlab # [ 216.580938] gitaly[1372]: time="2026-06-30T07:13:13.508Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H duration_ms=165ns method=POST pid=1372 status=200 url="http://unix/api/v4/internal/allowed"1936vm-test-run-nixbot-gitlab> gitlab # [ 216.612458] gitaly[1372]: time="2026-06-30T07:13:13.544Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H duration_ms=35ns method=POST pid=1372 status=200 url="http://unix/api/v4/internal/pre_receive"1937vm-test-run-nixbot-gitlab> gitlab # [ 216.619421] gitaly[1372]: time="2026-06-30T07:13:13.545Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:13.342" grpc.time_ms=202.841 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1938vm-test-run-nixbot-gitlab> gitlab # [ 216.679671] gitaly[1372]: time="2026-06-30T07:13:13.611Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:13.611" grpc.time_ms=0.553 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1939vm-test-run-nixbot-gitlab> gitlab # [ 216.692224] gitaly[1372]: time="2026-06-30T07:13:13.624Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZYACPEWD9CX0JGX57050 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:13.610" grpc.time_ms=14.34 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc1940vm-test-run-nixbot-gitlab> gitlab # [ 216.721658] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1051941vm-test-run-nixbot-gitlab> gitlab # [ 216.748966] gitaly[1372]: time="2026-06-30T07:13:13.680Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:13.679" grpc.time_ms=0.466 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1942vm-test-run-nixbot-gitlab> gitlab # [ 216.799325] gitaly[1372]: time="2026-06-30T07:13:13.731Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:13.731" grpc.time_ms=0.453 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1943vm-test-run-nixbot-gitlab> gitlab # [ 216.847255] gitaly[1372]: time="2026-06-30T07:13:13.779Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZYDQJKND2FCJRMSQVM38 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:13.763" pid=1372 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1944vm-test-run-nixbot-gitlab> gitlab # [ 216.847910] gitaly[1372]: time="2026-06-30T07:13:13.779Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZYDQJKND2FCJRMSQVM38 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:13.763" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc1945vm-test-run-nixbot-gitlab> gitlab # [ 216.886572] gitaly[1372]: time="2026-06-30T07:13:13.819Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZYDQJKND2FCJRMSQVM38 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:13.763" grpc.time_ms=55.721 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc1946vm-test-run-nixbot-gitlab> gitlab # [ 216.887487] gitlab-workhorse[1280]: time="2026-06-30T07:13:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491947vm-test-run-nixbot-gitlab> gitlab # [ 216.888318] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1641948vm-test-run-nixbot-gitlab> gitlab # [ 216.926394] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:13.857Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","e6cf5e803699fa6869c51ec2f1ff941148940ff0 f9f27455b0fb535a117810be3b433070f1df3310 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"bbecf4fcbadaf4716f565654","created_at":"2026-06-30T07:13:13.846Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-4840afe279df431e","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:13.853Z","job_size_bytes":185,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-bbecf4fcbadaf4716f565654: start","job_status":"start","queue_duration_s":0.003991,"scheduling_latency_s":0.004039}1949vm-test-run-nixbot-gitlab> gitlab # [ 216.958235] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:13.890Z","message":"changes: \"e6cf5e803699fa6869c51ec2f1ff941148940ff0 f9f27455b0fb535a117810be3b433070f1df3310 refs/heads/master\\n\""}1950vm-test-run-nixbot-gitlab> gitlab # [ 216.965888] gitaly[1372]: time="2026-06-30T07:13:13.897Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H duration_ms=107ns method=POST pid=1372 status=200 url="http://unix/api/v4/internal/post_receive"1951vm-test-run-nixbot-gitlab> gitlab # [ 216.966929] gitaly[1372]: time="2026-06-30T07:13:13.898Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:13.789" grpc.time_ms=108.704 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1952vm-test-run-nixbot-gitlab> gitlab # [ 217.006861] gitaly[1372]: time="2026-06-30T07:13:13.938Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:23.277" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:13.927" grpc.time_ms=11.091 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1953vm-test-run-nixbot-gitlab> gitlab # [ 217.008311] gitaly[1372]: time="2026-06-30T07:13:13.940Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H diskcache=6343ae77-2652-4a4e-ad3c-5923b6f12e83 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:13.263" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1954vm-test-run-nixbot-gitlab> gitlab # [ 217.009386] gitaly[1372]: time="2026-06-30T07:13:13.941Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=194 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=12998 command.oublock=32 command.real_time_ms=665 command.spawn_token_fork_ms=2 command.system_time_ms=148 command.user_time_ms=46 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=604 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:13.263" grpc.time_ms=677.902 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1955vm-test-run-nixbot-gitlab> gitlab # [ 217.013210] gitlab-workhorse[1280]: time="2026-06-30T07:13:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521956vm-test-run-nixbot-gitlab> gitlab # [ 217.014363] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:13 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 7531957vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1958vm-test-run-nixbot-gitlab> gitlab # e6cf5e8..f9f2745 master -> master1959vm-test-run-nixbot-gitlab> gitlab # [ 217.028575] gitaly[1372]: time="2026-06-30T07:13:13.960Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-30T07:13:23.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:13.951" grpc.time_ms=9.073 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960vm-test-run-nixbot-gitlab> gitlab # [ 217.064691] gitaly[1372]: time="2026-06-30T07:13:13.997Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-30T07:13:23.038" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:13.979" grpc.time_ms=17.31 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1961vm-test-run-nixbot-gitlab> gitlab # [ 217.082114] gitaly[1372]: time="2026-06-30T07:13:14.013Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-30T07:13:44.002" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:14.002" grpc.time_ms=10.813 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962vm-test-run-nixbot-gitlab> gitlab # [ 217.096539] gitaly[1372]: time="2026-06-30T07:13:14.029Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=4 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-30T07:13:24.002" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-30T07:13:14.023" grpc.time_ms=5.725 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963vm-test-run-nixbot-gitlab> gitlab # [ 217.118110] gitaly[1372]: time="2026-06-30T07:13:14.050Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:24.017" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:14.037" grpc.time_ms=11.839 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1964vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.16 seconds)1965vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1966vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1967vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f9f27455b0fb535a117810be3b433070f1df3310'1968vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f9f27455b0fb535a117810be3b433070f1df3310', in 0.02 seconds)1969vm-test-run-nixbot-gitlab> []1970vm-test-run-nixbot-gitlab> gitlab # [ 217.225508] gitaly[1372]: time="2026-06-30T07:13:14.155Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZYK0BH32YR5DEP6EQR69 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:14.136" pid=1372 remote_ip="2001:db8:1::1" request_sha=8e42123a9a4127845a002acd9c5c9ca686802ceb response_bytes=597 span.kind=server system=grpc1971vm-test-run-nixbot-gitlab> gitlab # [ 217.226442] gitlab-workhorse[1280]: time="2026-06-30T07:13:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=5971972vm-test-run-nixbot-gitlab> gitlab # [ 217.227715] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 597 "" "git/2.54.0" 3331973vm-test-run-nixbot-gitlab> gitlab # [ 217.229056] gitaly[1372]: time="2026-06-30T07:13:14.155Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZYK0BH32YR5DEP6EQR69 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:14.136" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc1974vm-test-run-nixbot-gitlab> gitlab # [ 217.230126] gitaly[1372]: time="2026-06-30T07:13:14.155Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZYK0BH32YR5DEP6EQR69 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:14.136" grpc.time_ms=18.939 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc1975vm-test-run-nixbot-gitlab> gitlab # [ 217.237818] nixbot[3192]: INFO: ::ffff:127.0.0.1:58676 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f9f27455b0fb535a117810be3b433070f1df3310 HTTP/1.1" 200 OK1976vm-test-run-nixbot-gitlab> gitlab # [ 217.282119] gitaly[1372]: time="2026-06-30T07:13:14.214Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-30T07:13:24.192" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:14.212" grpc.time_ms=1.443 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977vm-test-run-nixbot-gitlab> gitlab # [ 217.302906] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:14.235Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","e6cf5e803699fa6869c51ec2f1ff941148940ff0 f9f27455b0fb535a117810be3b433070f1df3310 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"bbecf4fcbadaf4716f565654","created_at":"2026-06-30T07:13:13.846Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-4840afe279df431e","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-30T07:13:13.853Z","job_size_bytes":185,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-bbecf4fcbadaf4716f565654: done: 0.377739 sec","job_status":"done","queue_duration_s":0.003991,"scheduling_latency_s":0.004039,"gitaly_calls":7,"gitaly_duration_s":0.09366,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.046498000000000005,"redis_read_bytes":350,"redis_write_bytes":13069,"redis_cache_calls":9,"redis_cache_duration_s":0.002205,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001405,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6769,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.005172,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.028615,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008636,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.191849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000733333,"exclusive_lock_hold_duration_s":0.041524552,"duration_s":0.377739,"completed_at":"2026-06-30T07:13:14.234Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02735,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1978vm-test-run-nixbot-gitlab> gitlab # [ 217.379262] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:14.236Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b08f4590ac2f083213753ecf","created_at":"2026-06-30T07:13:13.965Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-8baa80a261444ce8","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:13.967Z","job_size_bytes":46,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b08f4590ac2f083213753ecf: start","job_status":"start","queue_duration_s":0.269554,"scheduling_latency_s":0.269586}1979vm-test-run-nixbot-gitlab> gitlab # [ 217.380805] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:14.256Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b08f4590ac2f083213753ecf","created_at":"2026-06-30T07:13:13.965Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-8baa80a261444ce8","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:13.967Z","job_size_bytes":46,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b08f4590ac2f083213753ecf: done: 0.019508 sec","job_status":"done","queue_duration_s":0.269554,"scheduling_latency_s":0.269586,"redis_calls":9,"redis_duration_s":0.002043,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000248,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019508,"completed_at":"2026-06-30T07:13:14.256Z","load_balancing_strategy":"primary","db_duration_s":0.001749,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1980vm-test-run-nixbot-gitlab> gitlab # [ 217.470095] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:14.257Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c9ca33dd1e568ff73c1b14ae","created_at":"2026-06-30T07:13:13.973Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-8baa80a261444ce8","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:13.974Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c9ca33dd1e568ff73c1b14ae: start","job_status":"start","queue_duration_s":0.28269,"scheduling_latency_s":0.282712}1981vm-test-run-nixbot-gitlab> gitlab # [ 217.471386] gitaly[1372]: time="2026-06-30T07:13:14.300Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H diskcache=827e4f3d-47be-446a-8518-513bff36ad3f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T13:13:14.273" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:14.273" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982vm-test-run-nixbot-gitlab> gitlab # [ 217.472241] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:14.324Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c9ca33dd1e568ff73c1b14ae","created_at":"2026-06-30T07:13:13.973Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-8baa80a261444ce8","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:13.974Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c9ca33dd1e568ff73c1b14ae: done: 0.067238 sec","job_status":"done","queue_duration_s":0.28269,"scheduling_latency_s":0.282712,"gitaly_calls":1,"gitaly_duration_s":0.029413,"redis_calls":12,"redis_duration_s":0.0077150000000000005,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002811,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000686,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004218,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.045436501,"duration_s":0.067238,"completed_at":"2026-06-30T07:13:14.324Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007022,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1983vm-test-run-nixbot-gitlab> gitlab # [ 217.573470] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:14.328Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"bb1b42006eeddbc43b2139b0","created_at":"2026-06-30T07:13:14.089Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-8baa80a261444ce8","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0dc3be050e671370921127454a190bdfcd5bc87b2833c042fa65887f58b2f9ba","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:14.093Z","job_size_bytes":467,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-bb1b42006eeddbc43b2139b0: start","job_status":"start","queue_duration_s":0.235238,"scheduling_latency_s":0.235263}1984vm-test-run-nixbot-gitlab> gitlab # [ 217.575109] gitaly[1372]: time="2026-06-30T07:13:14.300Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1646 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T13:13:14.273" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:14.273" grpc.time_ms=26.454 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985vm-test-run-nixbot-gitlab> gitlab # [ 217.642762] gitaly[1372]: time="2026-06-30T07:13:14.350Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:44.349" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:14.350" grpc.time_ms=0.683 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986vm-test-run-nixbot-gitlab> gitlab # [ 217.644282] gitaly[1372]: time="2026-06-30T07:13:14.360Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-30T07:13:24.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:14.354" grpc.time_ms=6.65 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987vm-test-run-nixbot-gitlab> gitlab # [ 217.645190] gitaly[1372]: time="2026-06-30T07:13:14.388Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-30T07:13:44.386" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:14.386" grpc.time_ms=1.509 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988vm-test-run-nixbot-gitlab> gitlab # [ 217.646075] gitaly[1372]: time="2026-06-30T07:13:14.394Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=203 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-30T07:13:44.390" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:14.391" grpc.time_ms=2.625 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1989vm-test-run-nixbot-gitlab> gitlab # [ 217.726380] gitaly[1372]: time="2026-06-30T07:13:14.445Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:24.088" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:14.438" grpc.time_ms=6.315 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990vm-test-run-nixbot-gitlab> gitlab # [ 217.837922] gitaly[1372]: time="2026-06-30T07:13:14.770Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZEVATB1501FFNY33D2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:14.760" grpc.time_ms=10.06 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc1991vm-test-run-nixbot-gitlab> gitlab # [ 217.839397] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 551992vm-test-run-nixbot-gitlab> gitlab # [ 218.048842] gitaly[1372]: time="2026-06-30T07:13:14.981Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZHD4RR1W3PFE3QH9EVF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:14.968" pid=1372 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1993vm-test-run-nixbot-gitlab> gitlab # [ 218.050441] gitaly[1372]: time="2026-06-30T07:13:14.981Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZHD4RR1W3PFE3QH9EVF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:14.968" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc1994vm-test-run-nixbot-gitlab> gitlab # [ 218.052990] gitaly[1372]: time="2026-06-30T07:13:14.983Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZHD4RR1W3PFE3QH9EVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:14.968" grpc.time_ms=14.004 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc1995vm-test-run-nixbot-gitlab> gitlab # [ 218.054089] gitlab-workhorse[1280]: time="2026-06-30T07:13:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671996vm-test-run-nixbot-gitlab> gitlab # [ 218.056106] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 1861997vm-test-run-nixbot-gitlab> gitlab # [ 218.179485] gitaly[1372]: time="2026-06-30T07:13:15.111Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=970 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-30T07:13:45.099" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:15.100" grpc.time_ms=11.205 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1998vm-test-run-nixbot-gitlab> gitlab # [ 218.216923] gitaly[1372]: time="2026-06-30T07:13:15.149Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZXZ6QJJ3PFX7D1E8H56H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-30T07:13:45.138" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:15.139" grpc.time_ms=10.024 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1999vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f9f27455b0fb535a117810be3b433070f1df3310'2000vm-test-run-nixbot-gitlab> gitlab # [ 218.238879] gitaly[1372]: time="2026-06-30T07:13:15.169Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:45.167" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:15.168" grpc.time_ms=0.619 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2001vm-test-run-nixbot-gitlab> gitlab # [ 218.240281] sidekiq-cluster[2141]: {"severity":"WARN","time":"2026-06-30T07:13:15.161Z","class":"CreatePipelineWorker","correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"e6cf5e803699fa6869c51ec2f1ff941148940ff0","after":"f9f27455b0fb535a117810be3b433070f1df3310","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"f9f27455b0fb535a117810be3b433070f1df3310","ref":"refs/heads/master"},"retry":0}2002vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f9f27455b0fb535a117810be3b433070f1df3310', in 0.03 seconds)2003vm-test-run-nixbot-gitlab> []2004vm-test-run-nixbot-gitlab> gitlab # [ 218.241073] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:15.166Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"bb1b42006eeddbc43b2139b0","created_at":"2026-06-30T07:13:14.089Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-8baa80a261444ce8","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0dc3be050e671370921127454a190bdfcd5bc87b2833c042fa65887f58b2f9ba","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:14.093Z","job_size_bytes":467,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-bb1b42006eeddbc43b2139b0: done: 0.838669 sec","job_status":"done","queue_duration_s":0.235238,"scheduling_latency_s":0.235263,"gitaly_calls":7,"gitaly_duration_s":0.080915,"redis_calls":33,"redis_duration_s":0.00794,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000745,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004816,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.328,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.325,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.418543,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.838669,"completed_at":"2026-06-30T07:13:15.166Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.327704,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2005vm-test-run-nixbot-gitlab> gitlab # [ 218.316402] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:15.169Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"1df84313-7a04-4db6-b216-38ae4a2f6a43\"}"],"class":"WebHookWorker","jid":"05b9a22da3b3731f7f3e5870","created_at":"2026-06-30T07:13:14.196Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-8baa80a261444ce8","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9c086639642cb68acffbd2407f13bba6d9f49733113f5e5cb9c8bef19c8b8609","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:14.202Z","job_size_bytes":1749,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-05b9a22da3b3731f7f3e5870: start","job_status":"start","queue_duration_s":0.966639,"scheduling_latency_s":0.966685}2006vm-test-run-nixbot-gitlab> gitlab # [ 218.317963] gitaly[1372]: time="2026-06-30T07:13:15.197Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-30T07:13:45.188" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:15.189" grpc.time_ms=8.393 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2007vm-test-run-nixbot-gitlab> gitlab # [ 218.324575] nixbot[3192]: INFO: ::ffff:127.0.0.1:58692 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f9f27455b0fb535a117810be3b433070f1df3310 HTTP/1.1" 200 OK2008vm-test-run-nixbot-gitlab> gitlab # [ 218.395779] gitaly[1372]: time="2026-06-30T07:13:15.216Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:25.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:15.210" grpc.time_ms=6.067 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2009vm-test-run-nixbot-gitlab> gitlab # [ 218.398389] gitaly[1372]: time="2026-06-30T07:13:15.262Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-30T07:13:25.061" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:15.261" grpc.time_ms=0.23 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2010vm-test-run-nixbot-gitlab> gitlab # [ 218.399670] gitaly[1372]: time="2026-06-30T07:13:15.275Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-30T07:13:25.068" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:15.269" grpc.time_ms=6.464 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2011vm-test-run-nixbot-gitlab> gitlab # [ 218.401171] gitaly[1372]: time="2026-06-30T07:13:15.311Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:15.309" grpc.time_ms=1.545 pid=1372 span.kind=server system=grpc2012vm-test-run-nixbot-gitlab> gitlab # [ 218.403047] gitaly[1372]: time="2026-06-30T07:13:15.335Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:15.335" grpc.time_ms=0.439 pid=1372 span.kind=server system=grpc2013vm-test-run-nixbot-gitlab> gitlab # [ 218.475359] gitaly[1372]: time="2026-06-30T07:13:15.347Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D diskcache=545cc039-3dc4-45c6-89f3-1f18c3527751 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-30T07:13:25.077" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:15.277" pid=1372 ref=refs/keep-around/e6cf5e803699fa6869c51ec2f1ff941148940ff0 remote_ip="2001:db8:1::1" revision=e6cf5e803699fa6869c51ec2f1ff941148940ff0 span.kind=server system=grpc username=root2014vm-test-run-nixbot-gitlab> gitlab # [ 218.476793] gitaly[1372]: time="2026-06-30T07:13:15.348Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=59 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5358 command.oublock=8 command.real_time_ms=61 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-30T07:13:25.077" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:15.277" grpc.time_ms=70.029 pid=1372 ref=refs/keep-around/e6cf5e803699fa6869c51ec2f1ff941148940ff0 remote_ip="2001:db8:1::1" revision=e6cf5e803699fa6869c51ec2f1ff941148940ff0 span.kind=server system=grpc username=root2015vm-test-run-nixbot-gitlab> gitlab # [ 218.477941] gitaly[1372]: time="2026-06-30T07:13:15.352Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:45.351" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:15.352" grpc.time_ms=0.417 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2016vm-test-run-nixbot-gitlab> gitlab # [ 218.547509] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e6cf5e803699fa6869c51ec2f1ff941148940ff0 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 3742017vm-test-run-nixbot-gitlab> gitlab # [ 218.549056] nixbot[3192]: {"ts": "2026-06-30T07:13:15.480Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e6cf5e803699fa6869c51ec2f1ff941148940ff0 \"HTTP/1.1 201 Created\""}2018vm-test-run-nixbot-gitlab> gitlab # [ 218.555691] nixbot[3192]: {"ts": "2026-06-30T07:13:15.488Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-e6cf5e803699-df6c07c0", "argv0": "bwrap"}2019vm-test-run-nixbot-gitlab> gitlab # [ 218.639285] systemd[1]: Started Nix Daemon.2020vm-test-run-nixbot-gitlab> gitlab # [ 218.654471] nixbot[3192]: INFO: ::ffff:127.0.0.1:58694 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2021vm-test-run-nixbot-gitlab> gitlab # [ 218.672793] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:15.604Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"1df84313-7a04-4db6-b216-38ae4a2f6a43\"}"],"class":"WebHookWorker","jid":"05b9a22da3b3731f7f3e5870","created_at":"2026-06-30T07:13:14.196Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-8baa80a261444ce8","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9c086639642cb68acffbd2407f13bba6d9f49733113f5e5cb9c8bef19c8b8609","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:14.202Z","job_size_bytes":1749,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-05b9a22da3b3731f7f3e5870: done: 0.435303 sec","job_status":"done","queue_duration_s":0.966639,"scheduling_latency_s":0.966685,"redis_calls":19,"redis_duration_s":0.006335,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001066,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000391,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00325,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001628,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.0096339060000048,"cpu_s":0.073468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038273,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.435303,"completed_at":"2026-06-30T07:13:15.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2022vm-test-run-nixbot-gitlab> gitlab # [ 218.679235] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:15.607Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a8176f2b99e513ae72a87e25","created_at":"2026-06-30T07:13:15.356Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-347c21eb81cc46a4","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:15.363Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a8176f2b99e513ae72a87e25: start","job_status":"start","queue_duration_s":0.243627,"scheduling_latency_s":0.243651}2023vm-test-run-nixbot-gitlab> gitlab # [ 218.749151] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:15.652Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a8176f2b99e513ae72a87e25","created_at":"2026-06-30T07:13:15.356Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-347c21eb81cc46a4","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:15.363Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a8176f2b99e513ae72a87e25: done: 0.045622 sec","job_status":"done","queue_duration_s":0.243627,"scheduling_latency_s":0.243651,"redis_calls":18,"redis_duration_s":0.004247,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002314,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000366527,"duration_s":0.045622,"completed_at":"2026-06-30T07:13:15.652Z","load_balancing_strategy":"primary","db_duration_s":0.00824,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2024vm-test-run-nixbot-gitlab> gitlab # [ 218.776685] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:15.654Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"744fa265212adacdfd54d9ac","created_at":"2026-06-30T07:13:15.364Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-347c21eb81cc46a4","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:15.367Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-744fa265212adacdfd54d9ac: start","job_status":"start","queue_duration_s":0.28667,"scheduling_latency_s":0.286693}2025vm-test-run-nixbot-gitlab> gitlab # [ 218.819324] nix-daemon[3480]: accepted connection from pid 3478, user nixbot2026vm-test-run-nixbot-gitlab> gitlab # [ 218.826409] nix-daemon[3480]: accepted connection from pid 3479, user nixbot2027vm-test-run-nixbot-gitlab> gitlab # [ 218.886531] gitaly[1372]: time="2026-06-30T07:13:15.819Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=61 command.spawn_token_fork_ms=56 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBP00CME8QAM62VDP2A3HFW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:15.755" grpc.time_ms=63.38 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc2028vm-test-run-nixbot-gitlab> gitlab # [ 218.908922] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1722029vm-test-run-nixbot-gitlab> gitlab # [ 218.942522] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:15.874Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"744fa265212adacdfd54d9ac","created_at":"2026-06-30T07:13:15.364Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-347c21eb81cc46a4","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:15.367Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-744fa265212adacdfd54d9ac: done: 0.220389 sec","job_status":"done","queue_duration_s":0.28667,"scheduling_latency_s":0.286693,"redis_calls":15,"redis_duration_s":0.023027000000000002,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001138,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004927,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.016962,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.097,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.047,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001013257,"duration_s":0.220389,"completed_at":"2026-06-30T07:13:15.874Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.102768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030vm-test-run-nixbot-gitlab> gitlab # [ 219.016515] gitaly[1372]: time="2026-06-30T07:13:15.924Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00J3FK7EG07EVBGM79GQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:15.914" pid=1372 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2031vm-test-run-nixbot-gitlab> gitlab # [ 219.033129] gitlab-workhorse[1280]: time="2026-06-30T07:13:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672032vm-test-run-nixbot-gitlab> gitlab # [ 219.033922] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 832033vm-test-run-nixbot-gitlab> gitlab # [ 219.034566] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:15.895Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"44cc21c45efe07bd6e39598f","created_at":"2026-06-30T07:13:15.460Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-347c21eb81cc46a4","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:15.465Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-44cc21c45efe07bd6e39598f: start","job_status":"start","queue_duration_s":0.430084,"scheduling_latency_s":0.430161}2034vm-test-run-nixbot-gitlab> gitlab # [ 219.035477] gitaly[1372]: time="2026-06-30T07:13:15.924Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00J3FK7EG07EVBGM79GQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:15.914" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc2035vm-test-run-nixbot-gitlab> gitlab # [ 219.036360] gitaly[1372]: time="2026-06-30T07:13:15.927Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00J3FK7EG07EVBGM79GQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:15.914" grpc.time_ms=11.144 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc2036vm-test-run-nixbot-gitlab> gitlab # [ 219.132307] nixbot[3192]: {"ts": "2026-06-30T07:13:16.064Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}2037vm-test-run-nixbot-gitlab> gitlab # [ 219.147417] nix-daemon[3490]: unexpected Nix daemon error: error: interrupted by the user2038vm-test-run-nixbot-gitlab> gitlab # [ 219.176441] gitaly[1372]: time="2026-06-30T07:13:16.108Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-30T07:13:46.107" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:16.107" grpc.time_ms=0.791 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2039vm-test-run-nixbot-gitlab> gitlab # [ 219.222192] gitaly[1372]: time="2026-06-30T07:13:16.154Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00T1V0BJKQTXZ5G86T33 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:46.152" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:16.152" grpc.time_ms=1.786 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2040vm-test-run-nixbot-gitlab> gitlab # [ 219.250303] gitaly[1372]: time="2026-06-30T07:13:16.182Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:46.180" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:16.181" grpc.time_ms=0.666 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f9f27455b0fb535a117810be3b433070f1df3310'2042vm-test-run-nixbot-gitlab> gitlab # [ 219.271185] gitaly[1372]: time="2026-06-30T07:13:16.195Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBP00T1V0BJKQTXZ5G86T33 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:26.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:16.187" grpc.time_ms=7.756 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2043vm-test-run-nixbot-gitlab> gitlab # [ 219.295311] gitaly[1372]: time="2026-06-30T07:13:16.217Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:46.215" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:16.215" grpc.time_ms=1.652 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2044vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f9f27455b0fb535a117810be3b433070f1df3310', in 0.05 seconds)2045vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-06-30T07:13:16.056743Z', 'started_at': None, 'finished_at': None}]2046vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.29 seconds)2047vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses2048vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2049vm-test-run-nixbot-gitlab> gitlab # [ 219.315659] gitaly[1372]: time="2026-06-30T07:13:16.240Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=15 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-30T07:13:46.219" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:16.220" grpc.time_ms=20.03 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2050vm-test-run-nixbot-gitlab> gitlab # [ 219.338348] gitaly[1372]: time="2026-06-30T07:13:16.264Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:46.263" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:16.263" grpc.time_ms=0.591 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2051vm-test-run-nixbot-gitlab> gitlab # [ 219.358754] nixbot[3192]: INFO: ::ffff:127.0.0.1:58702 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f9f27455b0fb535a117810be3b433070f1df3310 HTTP/1.1" 200 OK2052vm-test-run-nixbot-gitlab> gitlab # [ 219.362153] nixbot[3192]: {"ts": "2026-06-30T07:13:16.277Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e6cf5e803699fa6869c51ec2f1ff941148940ff0 \"HTTP/1.1 201 Created\""}2053vm-test-run-nixbot-gitlab> gitlab # [ 219.366982] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e6cf5e803699fa6869c51ec2f1ff941148940ff0 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 1782054vm-test-run-nixbot-gitlab> gitlab # [ 219.371875] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:16.272Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"b562ec06a17cb1250f845e1d","created_at":"2026-06-30T07:13:16.268Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-2c26ba5a79194ecc","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"44cc21c45efe07bd6e39598f","job_size_bytes":3,"pid":2141,"job_status":"deduplicated","message":"PipelineProcessWorker JID-b562ec06a17cb1250f845e1d: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2055vm-test-run-nixbot-gitlab> gitlab # [ 219.396711] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:16.282Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"44cc21c45efe07bd6e39598f","created_at":"2026-06-30T07:13:15.460Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-347c21eb81cc46a4","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:15.465Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2141,"job_status":"rescheduled","message":"PipelineProcessWorker JID-44cc21c45efe07bd6e39598f: rescheduled"}2056vm-test-run-nixbot-gitlab> gitlab # [ 219.420923] gitaly[1372]: time="2026-06-30T07:13:16.271Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=8 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:26.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:16.257" grpc.time_ms=14.078 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2057vm-test-run-nixbot-gitlab> gitlab # [ 219.443110] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:16.316Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"44cc21c45efe07bd6e39598f","created_at":"2026-06-30T07:13:15.460Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-347c21eb81cc46a4","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:15.465Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-44cc21c45efe07bd6e39598f: done: 0.420453 sec","job_status":"done","queue_duration_s":0.430084,"scheduling_latency_s":0.430161,"gitaly_calls":3,"gitaly_duration_s":0.035187,"redis_calls":54,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.053701,"redis_read_bytes":292,"redis_write_bytes":8550,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.004392,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.015363,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.006299,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":4,"redis_queues_duration_s":0.001001,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2220,"redis_queues_metadata_calls":12,"redis_queues_metadata_duration_s":0.016622,"redis_queues_metadata_read_bytes":89,"redis_queues_metadata_write_bytes":1500,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.003192,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000678,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001762,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.139,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.079,"db_ci_duration_s":0.06,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.151083,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.29354568799999997,"duration_s":0.420453,"completed_at":"2026-06-30T07:13:16.316Z","load_balancing_strategy":"primary","db_duration_s":0.152476,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2058vm-test-run-nixbot-gitlab> gitlab # [ 219.526532] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:16.320Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","05b9a22da3b3731f7f3e5870"],"class":"WebHooks::LogExecutionWorker","jid":"425c8191e302f04c1e77b21c","created_at":"2026-06-30T07:13:15.590Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-c6f285d5a5b3447a","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da1b5361044af934fc637e503f19e9beda0bcaec17ca92c3458354e1f0871c58","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:15.597Z","job_size_bytes":2507,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-425c8191e302f04c1e77b21c: start","job_status":"start","queue_duration_s":0.723167,"scheduling_latency_s":0.723194}2059vm-test-run-nixbot-gitlab> gitlab # [ 219.527962] gitaly[1372]: time="2026-06-30T07:13:16.344Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-30T07:13:26.043" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:16.344" grpc.time_ms=0.238 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2060vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.25 seconds)2061vm-test-run-nixbot-gitlab> []2062vm-test-run-nixbot-gitlab> gitlab # [ 219.529511] gitaly[1372]: time="2026-06-30T07:13:16.357Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=619 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-30T07:13:26.049" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:16.349" grpc.time_ms=7.238 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2063vm-test-run-nixbot-gitlab> gitlab # [ 219.530444] gitaly[1372]: time="2026-06-30T07:13:16.371Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00ZYKY4DXMT0S5CH7BXX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:46.369" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:16.369" grpc.time_ms=0.987 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2064vm-test-run-nixbot-gitlab> gitlab # [ 219.541345] gitaly[1372]: time="2026-06-30T07:13:16.408Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:16.404" grpc.time_ms=1.029 pid=1372 span.kind=server system=grpc2065vm-test-run-nixbot-gitlab> gitlab # [ 219.557478] gitaly[1372]: time="2026-06-30T07:13:16.436Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP01107GH0NYV681PKA391 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:46.434" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:16.434" grpc.time_ms=1.663 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2066vm-test-run-nixbot-gitlab> gitlab # [ 219.576994] gitaly[1372]: time="2026-06-30T07:13:16.443Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:16.442" grpc.time_ms=0.725 pid=1372 span.kind=server system=grpc2067vm-test-run-nixbot-gitlab> gitlab # [ 219.654305] gitaly[1372]: time="2026-06-30T07:13:16.445Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBP00ZYKY4DXMT0S5CH7BXX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:26.250" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:16.431" grpc.time_ms=14.119 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2068vm-test-run-nixbot-gitlab> gitlab # [ 219.655578] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:16.581Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","05b9a22da3b3731f7f3e5870"],"class":"WebHooks::LogExecutionWorker","jid":"425c8191e302f04c1e77b21c","created_at":"2026-06-30T07:13:15.590Z","trace_propagation_headers":{"sentry-trace":"bbcd6c0f4385420d83cea526c302ed20-c6f285d5a5b3447a","baggage":"sentry-trace_id=bbcd6c0f4385420d83cea526c302ed20,sentry-release=Unknown"},"correlation_id":"01KWBNZXZ6QJJ3PFX7D1E8H56H","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da1b5361044af934fc637e503f19e9beda0bcaec17ca92c3458354e1f0871c58","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:15.597Z","job_size_bytes":2507,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-425c8191e302f04c1e77b21c: done: 0.261377 sec","job_status":"done","queue_duration_s":0.723167,"scheduling_latency_s":0.723194,"redis_calls":10,"redis_duration_s":0.005369,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000692,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002934,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001743,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.028,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.028,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.123006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001533435,"duration_s":0.261377,"completed_at":"2026-06-30T07:13:16.581Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039213,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2069vm-test-run-nixbot-gitlab> gitlab # [ 219.736287] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:16.586Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"44617c298118fab03ca1bb30","created_at":"2026-06-30T07:13:16.075Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-2c26ba5a79194ecc","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.083Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-44617c298118fab03ca1bb30: start","job_status":"start","queue_duration_s":0.502826,"scheduling_latency_s":0.50287}2070vm-test-run-nixbot-gitlab> gitlab # [ 219.737825] nixbot[3192]: {"ts": "2026-06-30T07:13:16.513Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e6cf5e803699fa6869c51ec2f1ff941148940ff0 \"HTTP/1.1 201 Created\""}2071vm-test-run-nixbot-gitlab> gitlab # [ 219.740631] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 1812072vm-test-run-nixbot-gitlab> gitlab # [ 219.742215] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e6cf5e803699fa6869c51ec2f1ff941148940ff0 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 2262073vm-test-run-nixbot-gitlab> gitlab # [ 219.743110] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:16.644Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"44617c298118fab03ca1bb30","created_at":"2026-06-30T07:13:16.075Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-2c26ba5a79194ecc","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.083Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-44617c298118fab03ca1bb30: done: 0.05856 sec","job_status":"done","queue_duration_s":0.502826,"scheduling_latency_s":0.50287,"redis_calls":10,"redis_duration_s":0.009799,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002381,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001299886,"duration_s":0.05856,"completed_at":"2026-06-30T07:13:16.644Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01113,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2074vm-test-run-nixbot-gitlab> gitlab # [ 219.818983] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:16.649Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"e0c2c2821b3c9448c236a9e4","created_at":"2026-06-30T07:13:16.283Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-2c26ba5a79194ecc","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.287Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e0c2c2821b3c9448c236a9e4: start","job_status":"start","queue_duration_s":0.361534,"scheduling_latency_s":0.36156}2075vm-test-run-nixbot-gitlab> gitlab # [ 219.880838] gitaly[1372]: time="2026-06-30T07:13:16.813Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 diskcache=7162588b-8a99-4894-b47a-fce53d8cbec4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-30T07:13:26.059" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:16.360" pid=1372 ref=refs/keep-around/f9f27455b0fb535a117810be3b433070f1df3310 remote_ip="2001:db8:1::1" revision=f9f27455b0fb535a117810be3b433070f1df3310 span.kind=server system=grpc username=root2076vm-test-run-nixbot-gitlab> gitlab # [ 219.882743] gitaly[1372]: time="2026-06-30T07:13:16.815Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=62 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5145 command.oublock=8 command.real_time_ms=91 command.spawn_token_fork_ms=3 command.system_time_ms=41 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-30T07:13:26.059" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-30T07:13:16.360" grpc.time_ms=454.258 pid=1372 ref=refs/keep-around/f9f27455b0fb535a117810be3b433070f1df3310 remote_ip="2001:db8:1::1" revision=f9f27455b0fb535a117810be3b433070f1df3310 span.kind=server system=grpc username=root2077vm-test-run-nixbot-gitlab> gitlab # [ 219.886798] gitaly[1372]: time="2026-06-30T07:13:16.819Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:46.818" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:16.818" grpc.time_ms=0.362 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2078vm-test-run-nixbot-gitlab> gitlab # [ 219.916132] gitaly[1372]: time="2026-06-30T07:13:16.848Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:16.848" grpc.time_ms=0.377 pid=1372 span.kind=server system=grpc2079vm-test-run-nixbot-gitlab> gitlab # [ 219.940390] gitaly[1372]: time="2026-06-30T07:13:16.872Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:16.872" grpc.time_ms=0.46 pid=1372 span.kind=server system=grpc2080vm-test-run-nixbot-gitlab> gitlab # [ 219.967163] gitaly[1372]: time="2026-06-30T07:13:16.899Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:16.899" grpc.time_ms=0.487 pid=1372 span.kind=server system=grpc2081vm-test-run-nixbot-gitlab> gitlab # [ 219.977204] gitaly[1372]: time="2026-06-30T07:13:16.909Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D diskcache=24c37f3c-a030-49de-92fd-7e310e6696fb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:46.777" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:16.813" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2082vm-test-run-nixbot-gitlab> gitlab # [ 219.978442] gitaly[1372]: time="2026-06-30T07:13:16.910Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=94 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7324 command.oublock=0 command.real_time_ms=88 command.spawn_token_fork_ms=1 command.system_time_ms=58 command.user_time_ms=36 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:46.777" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:16.813" grpc.time_ms=96.423 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2083vm-test-run-nixbot-gitlab> gitlab # [ 220.157345] gitaly[1372]: time="2026-06-30T07:13:17.089Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-30T07:13:47.088" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:17.088" grpc.time_ms=0.709 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2084vm-test-run-nixbot-gitlab> gitlab # [ 220.162513] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f9f27455b0fb535a117810be3b433070f1df3310 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 10252085vm-test-run-nixbot-gitlab> gitlab # [ 220.164220] nixbot[3192]: {"ts": "2026-06-30T07:13:17.096Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f9f27455b0fb535a117810be3b433070f1df3310 \"HTTP/1.1 201 Created\""}2086vm-test-run-nixbot-gitlab> gitlab # [ 220.172164] nixbot[3192]: {"ts": "2026-06-30T07:13:17.103Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-f9f27455b0fb-97732bf2", "argv0": "bwrap"}2087vm-test-run-nixbot-gitlab> gitlab # [ 220.212883] gitaly[1372]: time="2026-06-30T07:13:17.145Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:47.143" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:17.144" grpc.time_ms=1.164 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2088vm-test-run-nixbot-gitlab> gitlab # [ 220.278787] nix-daemon[3480]: accepted connection from pid 3589, user nixbot2089vm-test-run-nixbot-gitlab> gitlab # [ 220.282794] nix-daemon[3480]: accepted connection from pid 3590, user nixbot2090vm-test-run-nixbot-gitlab> gitlab # [ 220.316209] gitaly[1372]: time="2026-06-30T07:13:17.248Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBNZZV0SQSZ2289MB4SGZ2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:47.246" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:17.247" grpc.time_ms=0.702 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2091vm-test-run-nixbot-gitlab> gitlab # [ 220.349260] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:17.281Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"e0c2c2821b3c9448c236a9e4","created_at":"2026-06-30T07:13:16.283Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-2c26ba5a79194ecc","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.287Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2141,"job_status":"rescheduled","message":"PipelineProcessWorker JID-e0c2c2821b3c9448c236a9e4: rescheduled"}2092vm-test-run-nixbot-gitlab> gitlab # [ 220.364328] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:17.296Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"e0c2c2821b3c9448c236a9e4","created_at":"2026-06-30T07:13:16.283Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-2c26ba5a79194ecc","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.287Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e0c2c2821b3c9448c236a9e4: done: 0.647101 sec","job_status":"done","queue_duration_s":0.361534,"scheduling_latency_s":0.36156,"gitaly_calls":4,"gitaly_duration_s":0.1815,"redis_calls":101,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.12142700000000001,"redis_read_bytes":186,"redis_write_bytes":23556,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.004077,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003148,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.021599,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.013883,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12388,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.060366,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001985,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002768,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009524,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.131,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.02,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.093,"db_ci_duration_s":0.038,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.181016,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.592994361,"duration_s":0.647101,"completed_at":"2026-06-30T07:13:17.296Z","load_balancing_strategy":"primary","db_duration_s":0.139093,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2093vm-test-run-nixbot-gitlab> gitlab # [ 220.445817] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:17.304Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"821607ac18960ccbfb2e5267","created_at":"2026-06-30T07:13:16.749Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.760Z","job_size_bytes":23,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-821607ac18960ccbfb2e5267: start","job_status":"start","queue_duration_s":0.543964,"scheduling_latency_s":0.543996}2094vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2095vm-test-run-nixbot-gitlab> gitlab # [ 220.587490] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:17.519Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"821607ac18960ccbfb2e5267","created_at":"2026-06-30T07:13:16.749Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.760Z","job_size_bytes":23,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-821607ac18960ccbfb2e5267: done: 0.215177 sec","job_status":"done","queue_duration_s":0.543964,"scheduling_latency_s":0.543996,"redis_calls":15,"redis_duration_s":0.016387,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004102,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.001203,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.006577,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001863,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002642,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098447,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001270552,"duration_s":0.215177,"completed_at":"2026-06-30T07:13:17.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039332,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2096vm-test-run-nixbot-gitlab> gitlab # [ 220.669281] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:17.601Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"aa23ea1f5f0eea97423b2c8c","created_at":"2026-06-30T07:13:16.823Z","trace_propagation_headers":{"sentry-trace":"76aefcc8c9f44c28a3f3a6fb4e7809f7-bc2575163774433f","baggage":"sentry-trace_id=76aefcc8c9f44c28a3f3a6fb4e7809f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP00S5V3YKD9NSN4XJXBD9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.958Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-aa23ea1f5f0eea97423b2c8c: start","job_status":"start","queue_duration_s":0.642777,"scheduling_latency_s":0.642824}2097vm-test-run-nixbot-gitlab> gitlab # [ 220.761346] gitaly[1372]: time="2026-06-30T07:13:17.693Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP029NF3TFJSV09R711RCZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:47.692" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:17.692" grpc.time_ms=0.981 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098vm-test-run-nixbot-gitlab> gitlab # [ 220.781624] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:17.708Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"aa23ea1f5f0eea97423b2c8c","created_at":"2026-06-30T07:13:16.823Z","trace_propagation_headers":{"sentry-trace":"76aefcc8c9f44c28a3f3a6fb4e7809f7-bc2575163774433f","baggage":"sentry-trace_id=76aefcc8c9f44c28a3f3a6fb4e7809f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP00S5V3YKD9NSN4XJXBD9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.958Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-aa23ea1f5f0eea97423b2c8c: done: 0.107619 sec","job_status":"done","queue_duration_s":0.642777,"scheduling_latency_s":0.642824,"redis_calls":7,"redis_duration_s":0.019094,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004342,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.107619,"completed_at":"2026-06-30T07:13:17.708Z","load_balancing_strategy":"primary","db_duration_s":0.008566,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2099vm-test-run-nixbot-gitlab> gitlab # [ 220.785295] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:17.717Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"bb0ea6237b9335b33d262652","created_at":"2026-06-30T07:13:16.917Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.964Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-bb0ea6237b9335b33d262652: start","job_status":"start","queue_duration_s":0.752842,"scheduling_latency_s":0.752899}2100vm-test-run-nixbot-gitlab> gitlab # [ 220.870779] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 992101vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.30 seconds)2102vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-30T07:13:17.066Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2103vm-test-run-nixbot-gitlab> gitlab # [ 220.910770] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:17.842Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"bb0ea6237b9335b33d262652","created_at":"2026-06-30T07:13:16.917Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.964Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-bb0ea6237b9335b33d262652: done: 0.12577 sec","job_status":"done","queue_duration_s":0.752842,"scheduling_latency_s":0.752899,"redis_calls":10,"redis_duration_s":0.008751,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003021,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003535,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002195,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.08,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.075,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001364419,"duration_s":0.12577,"completed_at":"2026-06-30T07:13:17.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.080057,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2104vm-test-run-nixbot-gitlab> gitlab # [ 220.946812] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:17.847Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"42d39fd5095939ba745d5072","created_at":"2026-06-30T07:13:17.504Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-b084c4f8402f48ae","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:74c0a0af5dad795bd89bab9c706a6bf3a819f445bad0ad4a332b660c753a6ad6","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.508Z","job_size_bytes":464,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-42d39fd5095939ba745d5072: start","job_status":"start","queue_duration_s":0.339168,"scheduling_latency_s":0.339193}2105vm-test-run-nixbot-gitlab> gitlab # [ 221.062655] sidekiq-cluster[2141]: {"severity":"WARN","time":"2026-06-30T07:13:17.993Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"42d39fd5095939ba745d5072","created_at":"2026-06-30T07:13:17.504Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-b084c4f8402f48ae","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:74c0a0af5dad795bd89bab9c706a6bf3a819f445bad0ad4a332b660c753a6ad6","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.508Z","job_size_bytes":464,"pid":2141,"sidekiq_tid":"m47x","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-42d39fd5095939ba745d5072: fail: 0.14494 sec","job_status":"fail","queue_duration_s":0.339168,"scheduling_latency_s":0.339193,"redis_calls":7,"redis_duration_s":0.008454,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004349,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004105,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.072,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.07,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.14494,"completed_at":"2026-06-30T07:13:17.992Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.072002,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106vm-test-run-nixbot-gitlab> gitlab # [ 221.678653] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:18.609Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"bc4fa7d4a2df6bbd757d2a85","created_at":"2026-06-30T07:13:16.960Z","trace_propagation_headers":{"sentry-trace":"76aefcc8c9f44c28a3f3a6fb4e7809f7-bc2575163774433f","baggage":"sentry-trace_id=76aefcc8c9f44c28a3f3a6fb4e7809f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP00S5V3YKD9NSN4XJXBD9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.964Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-bc4fa7d4a2df6bbd757d2a85: start","job_status":"start","queue_duration_s":1.64476,"scheduling_latency_s":1.644842}2107vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2108vm-test-run-nixbot-gitlab> gitlab # [ 221.908241] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:18.839Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"bc4fa7d4a2df6bbd757d2a85","created_at":"2026-06-30T07:13:16.960Z","trace_propagation_headers":{"sentry-trace":"76aefcc8c9f44c28a3f3a6fb4e7809f7-bc2575163774433f","baggage":"sentry-trace_id=76aefcc8c9f44c28a3f3a6fb4e7809f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP00S5V3YKD9NSN4XJXBD9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:16.964Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-bc4fa7d4a2df6bbd757d2a85: done: 0.230136 sec","job_status":"done","queue_duration_s":1.64476,"scheduling_latency_s":1.644842,"redis_calls":16,"redis_duration_s":0.043288,"redis_read_bytes":20,"redis_write_bytes":1690,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.019104,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":833,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.012832,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.230136,"completed_at":"2026-06-30T07:13:18.839Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2109vm-test-run-nixbot-gitlab> gitlab # [ 222.008355] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:18.940Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ec1bc18ea8ae301f9589f966","created_at":"2026-06-30T07:13:16.990Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.007Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ec1bc18ea8ae301f9589f966: start","job_status":"start","queue_duration_s":1.932701,"scheduling_latency_s":1.932765}2110vm-test-run-nixbot-gitlab> gitlab # [ 222.052393] gitaly[1372]: time="2026-06-30T07:13:18.983Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP03GES7NMESY08XYP5QP0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:48.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:18.982" grpc.time_ms=0.902 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2111vm-test-run-nixbot-gitlab> gitlab # [ 222.078185] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1472112vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.20 seconds)2113vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-30T07:13:17.066Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2114vm-test-run-nixbot-gitlab> gitlab # [ 222.088490] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.020Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ec1bc18ea8ae301f9589f966","created_at":"2026-06-30T07:13:16.990Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.007Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ec1bc18ea8ae301f9589f966: done: 0.08026 sec","job_status":"done","queue_duration_s":1.932701,"scheduling_latency_s":1.932765,"redis_calls":10,"redis_duration_s":0.015799,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001842,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011833,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002124,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000395301,"duration_s":0.08026,"completed_at":"2026-06-30T07:13:19.020Z","load_balancing_strategy":"primary","db_duration_s":0.018083,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2115vm-test-run-nixbot-gitlab> gitlab # [ 222.125103] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.028Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"30033db377684c3dcaf2dad8","created_at":"2026-06-30T07:13:17.024Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.038Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-30033db377684c3dcaf2dad8: start","job_status":"start","queue_duration_s":1.989772,"scheduling_latency_s":1.989808}2116vm-test-run-nixbot-gitlab> gitlab # [ 222.228652] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.160Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"30033db377684c3dcaf2dad8","created_at":"2026-06-30T07:13:17.024Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.038Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-30033db377684c3dcaf2dad8: done: 0.131513 sec","job_status":"done","queue_duration_s":1.989772,"scheduling_latency_s":1.989808,"redis_calls":13,"redis_duration_s":0.072684,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001531,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011581,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.056885,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002687,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003252928,"duration_s":0.131513,"completed_at":"2026-06-30T07:13:19.160Z","load_balancing_strategy":"primary","db_duration_s":0.007831,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2117vm-test-run-nixbot-gitlab> gitlab # [ 222.311186] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.242Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"70ec6ad96fd7d9a3131174dd","created_at":"2026-06-30T07:13:17.040Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.047Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-70ec6ad96fd7d9a3131174dd: start","job_status":"start","queue_duration_s":2.194354,"scheduling_latency_s":2.19443}2118vm-test-run-nixbot-gitlab> gitlab # [ 222.444793] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.376Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"70ec6ad96fd7d9a3131174dd","created_at":"2026-06-30T07:13:17.040Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.047Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-70ec6ad96fd7d9a3131174dd: done: 0.134139 sec","job_status":"done","queue_duration_s":2.194354,"scheduling_latency_s":2.19443,"redis_calls":10,"redis_duration_s":0.027430000000000003,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007362,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01597,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004098,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00343647,"duration_s":0.134139,"completed_at":"2026-06-30T07:13:19.376Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119vm-test-run-nixbot-gitlab> gitlab # [ 222.552261] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.484Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9954bc408a8763d84313c7f4","created_at":"2026-06-30T07:13:17.049Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.060Z","job_size_bytes":19,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9954bc408a8763d84313c7f4: start","job_status":"start","queue_duration_s":2.422955,"scheduling_latency_s":2.423029}2120vm-test-run-nixbot-gitlab> gitlab # [ 222.659877] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.591Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9954bc408a8763d84313c7f4","created_at":"2026-06-30T07:13:17.049Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.060Z","job_size_bytes":19,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9954bc408a8763d84313c7f4: done: 0.10764 sec","job_status":"done","queue_duration_s":2.422955,"scheduling_latency_s":2.423029,"redis_calls":11,"redis_duration_s":0.027118,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00594,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.015044,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006134,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003368026,"duration_s":0.10764,"completed_at":"2026-06-30T07:13:19.591Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011864,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121vm-test-run-nixbot-gitlab> gitlab # [ 222.756582] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.687Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"83b25684945d4fda3eb200bf","created_at":"2026-06-30T07:13:17.063Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.072Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-83b25684945d4fda3eb200bf: start","job_status":"start","queue_duration_s":2.614578,"scheduling_latency_s":2.61465}2122vm-test-run-nixbot-gitlab> gitlab # [ 222.888910] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.820Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"83b25684945d4fda3eb200bf","created_at":"2026-06-30T07:13:17.063Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.072Z","job_size_bytes":65,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-83b25684945d4fda3eb200bf: done: 0.133537 sec","job_status":"done","queue_duration_s":2.614578,"scheduling_latency_s":2.61465,"redis_calls":10,"redis_duration_s":0.022432,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015935,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002397,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003281702,"duration_s":0.133537,"completed_at":"2026-06-30T07:13:19.820Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013085,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2123vm-test-run-nixbot-gitlab> gitlab # [ 223.002240] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:19.933Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"83dfb5227a7b2120b8c7cc32","created_at":"2026-06-30T07:13:17.073Z","trace_propagation_headers":{"sentry-trace":"76aefcc8c9f44c28a3f3a6fb4e7809f7-bc2575163774433f","baggage":"sentry-trace_id=76aefcc8c9f44c28a3f3a6fb4e7809f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP00S5V3YKD9NSN4XJXBD9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.077Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-83dfb5227a7b2120b8c7cc32: start","job_status":"start","queue_duration_s":2.855547,"scheduling_latency_s":2.855622}2124vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2125vm-test-run-nixbot-gitlab> gitlab # [ 223.200814] gitaly[1372]: time="2026-06-30T07:13:20.132Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP04P3AXC8K5S0QXWPRH7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:50.129" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:20.130" grpc.time_ms=1.011 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2126vm-test-run-nixbot-gitlab> gitlab # [ 223.214748] gitaly[1372]: time="2026-06-30T07:13:20.147Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-30T07:13:50.144" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:20.144" grpc.time_ms=2.501 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2127vm-test-run-nixbot-gitlab> gitlab # [ 223.228369] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 912128vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.17 seconds)2129vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-30T07:13:17.066Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2130vm-test-run-nixbot-gitlab> gitlab # [ 223.281088] gitaly[1372]: time="2026-06-30T07:13:20.213Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:50.211" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:20.212" grpc.time_ms=0.834 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2131vm-test-run-nixbot-gitlab> gitlab # [ 223.338418] gitaly[1372]: time="2026-06-30T07:13:20.270Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP00S5V3YKD9NSN4XJXBD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:50.269" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:20.269" grpc.time_ms=0.805 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2132vm-test-run-nixbot-gitlab> gitlab # [ 223.359531] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.291Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"83dfb5227a7b2120b8c7cc32","created_at":"2026-06-30T07:13:17.073Z","trace_propagation_headers":{"sentry-trace":"76aefcc8c9f44c28a3f3a6fb4e7809f7-bc2575163774433f","baggage":"sentry-trace_id=76aefcc8c9f44c28a3f3a6fb4e7809f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP00S5V3YKD9NSN4XJXBD9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.077Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-83dfb5227a7b2120b8c7cc32: done: 0.358244 sec","job_status":"done","queue_duration_s":2.855547,"scheduling_latency_s":2.855622,"gitaly_calls":3,"gitaly_duration_s":0.046093,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.04461,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.009055,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004992,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.003433,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.009196,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000398,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000993,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.085,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.014,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.044,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140708,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.260958306,"duration_s":0.358244,"completed_at":"2026-06-30T07:13:20.291Z","load_balancing_strategy":"primary","db_duration_s":0.099033,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2133vm-test-run-nixbot-gitlab> gitlab # [ 223.396755] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.296Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"900ea37a9030624dc267ab64","created_at":"2026-06-30T07:13:17.076Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.079Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-900ea37a9030624dc267ab64: start","job_status":"start","queue_duration_s":3.216927,"scheduling_latency_s":3.216954}2134vm-test-run-nixbot-gitlab> gitlab # [ 223.505190] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.437Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"900ea37a9030624dc267ab64","created_at":"2026-06-30T07:13:17.076Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.079Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-900ea37a9030624dc267ab64: done: 0.140791 sec","job_status":"done","queue_duration_s":3.216927,"scheduling_latency_s":3.216954,"redis_calls":7,"redis_duration_s":0.012071,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009794,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002277,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.087,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.085,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.140791,"completed_at":"2026-06-30T07:13:20.437Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.087104,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135vm-test-run-nixbot-gitlab> gitlab # [ 223.514740] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.447Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"26e60dfaf4a9c1ab148846e5","created_at":"2026-06-30T07:13:17.159Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.173Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-26e60dfaf4a9c1ab148846e5: start","job_status":"start","queue_duration_s":3.273599,"scheduling_latency_s":3.273632}2136vm-test-run-nixbot-gitlab> gitlab # [ 223.598229] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.530Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"26e60dfaf4a9c1ab148846e5","created_at":"2026-06-30T07:13:17.159Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.173Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-26e60dfaf4a9c1ab148846e5: done: 0.083229 sec","job_status":"done","queue_duration_s":3.273599,"scheduling_latency_s":3.273632,"redis_calls":10,"redis_duration_s":0.015495,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002953,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002269562,"duration_s":0.083229,"completed_at":"2026-06-30T07:13:20.530Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030207,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2137vm-test-run-nixbot-gitlab> gitlab # [ 223.605379] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.537Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a5a7d3cb0ce34757ad5463fc","created_at":"2026-06-30T07:13:17.175Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.183Z","job_size_bytes":6,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a5a7d3cb0ce34757ad5463fc: start","job_status":"start","queue_duration_s":3.354375,"scheduling_latency_s":3.354401}2138vm-test-run-nixbot-gitlab> gitlab # [ 223.651496] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.583Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a5a7d3cb0ce34757ad5463fc","created_at":"2026-06-30T07:13:17.175Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.183Z","job_size_bytes":6,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a5a7d3cb0ce34757ad5463fc: done: 0.046035 sec","job_status":"done","queue_duration_s":3.354375,"scheduling_latency_s":3.354401,"redis_calls":10,"redis_duration_s":0.011793999999999999,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002056,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002218,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001386489,"duration_s":0.046035,"completed_at":"2026-06-30T07:13:20.583Z","load_balancing_strategy":"primary","db_duration_s":0.006262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139vm-test-run-nixbot-gitlab> gitlab # [ 223.694278] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.586Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"038a5fd31c89904a5d36c87d","created_at":"2026-06-30T07:13:17.282Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.286Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-038a5fd31c89904a5d36c87d: start","job_status":"start","queue_duration_s":3.300685,"scheduling_latency_s":3.300709}2140vm-test-run-nixbot-gitlab> gitlab # [ 223.773169] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.704Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"038a5fd31c89904a5d36c87d","created_at":"2026-06-30T07:13:17.282Z","trace_propagation_headers":{"sentry-trace":"76dffdaaf9974b82a4fa0e3dd21a7709-44d581c8c1244a7e","baggage":"sentry-trace_id=76dffdaaf9974b82a4fa0e3dd21a7709,sentry-release=Unknown"},"correlation_id":"01KWBNZZV0SQSZ2289MB4SGZ2D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:17.286Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-038a5fd31c89904a5d36c87d: done: 0.117341 sec","job_status":"done","queue_duration_s":3.300685,"scheduling_latency_s":3.300709,"redis_calls":8,"redis_duration_s":0.010335,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00909,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001245,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.076,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.076,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.117341,"completed_at":"2026-06-30T07:13:20.704Z","load_balancing_strategy":"primary","db_duration_s":0.075996,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2141vm-test-run-nixbot-gitlab> gitlab # [ 223.785327] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.715Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"ce76482839fa0cd0e879e9d6","created_at":"2026-06-30T07:13:20.100Z","trace_propagation_headers":{"sentry-trace":"76aefcc8c9f44c28a3f3a6fb4e7809f7-caf7a7a097584573","baggage":"sentry-trace_id=76aefcc8c9f44c28a3f3a6fb4e7809f7,sentry-release=Unknown"},"correlation_id":"01KWBP00S5V3YKD9NSN4XJXBD9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:20.109Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ce76482839fa0cd0e879e9d6: start","job_status":"start","queue_duration_s":0.605636,"scheduling_latency_s":0.605681}2142vm-test-run-nixbot-gitlab> gitlab # [ 223.863614] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:20.795Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"ce76482839fa0cd0e879e9d6","created_at":"2026-06-30T07:13:20.100Z","trace_propagation_headers":{"sentry-trace":"76aefcc8c9f44c28a3f3a6fb4e7809f7-caf7a7a097584573","baggage":"sentry-trace_id=76aefcc8c9f44c28a3f3a6fb4e7809f7,sentry-release=Unknown"},"correlation_id":"01KWBP00S5V3YKD9NSN4XJXBD9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:20.109Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ce76482839fa0cd0e879e9d6: done: 0.080455 sec","job_status":"done","queue_duration_s":0.605636,"scheduling_latency_s":0.605681,"redis_calls":7,"redis_duration_s":0.020465999999999998,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01134,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.009126,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.080455,"completed_at":"2026-06-30T07:13:20.795Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2144vm-test-run-nixbot-gitlab> gitlab # [ 224.390369] gitaly[1372]: time="2026-06-30T07:13:21.321Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP05TSQP3SA9NDXZRYSP4P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:51.319" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:21.320" grpc.time_ms=1.248 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2145vm-test-run-nixbot-gitlab> gitlab # [ 224.482156] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1712146vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.24 seconds)2147vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-30T07:13:17.066Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2148vm-test-run-nixbot-gitlab> gitlab # [ 225.235205] nix-daemon[3480]: accepted connection from pid 3624, user nixbot2149vm-test-run-nixbot-gitlab> gitlab # [ 225.273177] gitaly[1372]: time="2026-06-30T07:13:22.205Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:52.202" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:22.203" grpc.time_ms=1.538 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2150vm-test-run-nixbot-gitlab> gitlab # [ 225.362682] gitaly[1372]: time="2026-06-30T07:13:22.295Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:32.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:22.282" grpc.time_ms=12.456 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2151vm-test-run-nixbot-gitlab> gitlab # [ 225.422479] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:22 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f9f27455b0fb535a117810be3b433070f1df3310 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 2152152vm-test-run-nixbot-gitlab> gitlab # [ 225.423901] nixbot[3192]: {"ts": "2026-06-30T07:13:22.355Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f9f27455b0fb535a117810be3b433070f1df3310 \"HTTP/1.1 201 Created\""}2153vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2154vm-test-run-nixbot-gitlab> gitlab # [ 225.466344] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.394Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"3c543d8bbd9b23c06cd6d08e","created_at":"2026-06-30T07:13:22.333Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-9210c388ff6f466d","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.338Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3c543d8bbd9b23c06cd6d08e: start","job_status":"start","queue_duration_s":0.056311,"scheduling_latency_s":0.056454}2155vm-test-run-nixbot-gitlab> gitlab # [ 225.499756] gitaly[1372]: time="2026-06-30T07:13:22.431Z" level=info msg="finished unary call with code OK" catfile.duration_ms=17 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=17 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP06Y08B1KHSNTPYTY3Z4G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:52.413" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:22.414" grpc.time_ms=17.363 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2156vm-test-run-nixbot-gitlab> gitlab # [ 225.537297] gitaly[1372]: time="2026-06-30T07:13:22.469Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBP06Y08B1KHSNTPYTY3Z4G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:32.160" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:22.461" grpc.time_ms=8.451 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2157vm-test-run-nixbot-gitlab> gitlab # [ 225.588447] gitaly[1372]: time="2026-06-30T07:13:22.520Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0715X3NT65YTV7QTKTVB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:52.518" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:22.520" grpc.time_ms=0.633 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2158vm-test-run-nixbot-gitlab> gitlab # [ 225.609860] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:22 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f9f27455b0fb535a117810be3b433070f1df3310 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1662159vm-test-run-nixbot-gitlab> gitlab # [ 225.614503] nixbot[3192]: {"ts": "2026-06-30T07:13:22.535Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f9f27455b0fb535a117810be3b433070f1df3310 \"HTTP/1.1 201 Created\""}2160vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.14 seconds)2161vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-30T07:13:17.066Z', 'started_at': None, 'finished_at': '2026-06-30T07:13:22.325Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-30T07:13:22.515Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2162vm-test-run-nixbot-gitlab> gitlab # [ 225.626847] gitaly[1372]: time="2026-06-30T07:13:22.559Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:22.558" grpc.time_ms=0.336 pid=1372 span.kind=server system=grpc2163vm-test-run-nixbot-gitlab> gitlab # [ 225.642577] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 942164vm-test-run-nixbot-gitlab> gitlab # [ 225.664711] gitaly[1372]: time="2026-06-30T07:13:22.597Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:22.596" grpc.time_ms=0.552 pid=1372 span.kind=server system=grpc2165vm-test-run-nixbot-gitlab> gitlab # [ 225.705871] gitaly[1372]: time="2026-06-30T07:13:22.638Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:22.637" grpc.time_ms=0.424 pid=1372 span.kind=server system=grpc2166vm-test-run-nixbot-gitlab> gitlab # [ 225.731698] gitaly[1372]: time="2026-06-30T07:13:22.664Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M diskcache=c5da8712-0cdd-4f51-9167-cc94c17a8663 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:52.530" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:22.530" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2167vm-test-run-nixbot-gitlab> gitlab # [ 225.732335] gitaly[1372]: time="2026-06-30T07:13:22.664Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=105 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7301 command.oublock=0 command.real_time_ms=125 command.spawn_token_fork_ms=0 command.system_time_ms=61 command.user_time_ms=44 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:52.530" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:22.530" grpc.time_ms=133.692 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2168vm-test-run-nixbot-gitlab> gitlab # [ 225.778244] gitaly[1372]: time="2026-06-30T07:13:22.710Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-30T07:13:52.709" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:22.709" grpc.time_ms=0.933 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2169vm-test-run-nixbot-gitlab> gitlab # [ 225.800718] gitaly[1372]: time="2026-06-30T07:13:22.733Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:52.732" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:22.732" grpc.time_ms=0.709 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2170vm-test-run-nixbot-gitlab> gitlab # [ 225.845629] gitaly[1372]: time="2026-06-30T07:13:22.778Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:52.776" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:22.777" grpc.time_ms=0.942 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2171vm-test-run-nixbot-gitlab> gitlab # [ 225.864995] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.796Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"bfe6de6d35f0f931077368b6","created_at":"2026-06-30T07:13:22.792Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"3c543d8bbd9b23c06cd6d08e","job_size_bytes":3,"pid":2141,"job_status":"deduplicated","message":"PipelineProcessWorker JID-bfe6de6d35f0f931077368b6: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2172vm-test-run-nixbot-gitlab> gitlab # [ 225.868870] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.801Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"3c543d8bbd9b23c06cd6d08e","created_at":"2026-06-30T07:13:22.333Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-9210c388ff6f466d","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.338Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2141,"job_status":"rescheduled","message":"PipelineProcessWorker JID-3c543d8bbd9b23c06cd6d08e: rescheduled"}2173vm-test-run-nixbot-gitlab> gitlab # [ 225.882756] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.814Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"3c543d8bbd9b23c06cd6d08e","created_at":"2026-06-30T07:13:22.333Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-9210c388ff6f466d","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.338Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3c543d8bbd9b23c06cd6d08e: done: 0.419974 sec","job_status":"done","queue_duration_s":0.056311,"scheduling_latency_s":0.056454,"gitaly_calls":4,"gitaly_duration_s":0.147178,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.025697,"redis_read_bytes":262,"redis_write_bytes":23383,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001482,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003502,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001035,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.004522,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14464,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.00915,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000338,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000832,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003354,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.163625,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.310972484,"duration_s":0.419974,"completed_at":"2026-06-30T07:13:22.814Z","load_balancing_strategy":"primary","db_duration_s":0.035691,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2174vm-test-run-nixbot-gitlab> gitlab # [ 225.995144] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.817Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3689ed8b28e5fd9416050569","created_at":"2026-06-30T07:13:22.520Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.522Z","job_size_bytes":23,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3689ed8b28e5fd9416050569: start","job_status":"start","queue_duration_s":0.29509,"scheduling_latency_s":0.295195}2175vm-test-run-nixbot-gitlab> gitlab # [ 225.996682] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.862Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3689ed8b28e5fd9416050569","created_at":"2026-06-30T07:13:22.520Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.522Z","job_size_bytes":23,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3689ed8b28e5fd9416050569: done: 0.04442 sec","job_status":"done","queue_duration_s":0.29509,"scheduling_latency_s":0.295195,"redis_calls":7,"redis_duration_s":0.003083,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04442,"completed_at":"2026-06-30T07:13:22.862Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011323,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2176vm-test-run-nixbot-gitlab> gitlab # [ 226.080577] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.864Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2a7a0f7ea8184ada01f1814d","created_at":"2026-06-30T07:13:22.670Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.674Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2a7a0f7ea8184ada01f1814d: start","job_status":"start","queue_duration_s":0.189774,"scheduling_latency_s":0.189801}2177vm-test-run-nixbot-gitlab> gitlab # [ 226.082483] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.888Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2a7a0f7ea8184ada01f1814d","created_at":"2026-06-30T07:13:22.670Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.674Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2a7a0f7ea8184ada01f1814d: done: 0.024103 sec","job_status":"done","queue_duration_s":0.189774,"scheduling_latency_s":0.189801,"redis_calls":7,"redis_duration_s":0.0029560000000000003,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000796,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024103,"completed_at":"2026-06-30T07:13:22.888Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2178vm-test-run-nixbot-gitlab> gitlab # [ 226.160900] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.890Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"a9c4b0e7bdbbc2d258eb92af","created_at":"2026-06-30T07:13:22.683Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.685Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-a9c4b0e7bdbbc2d258eb92af: start","job_status":"start","queue_duration_s":0.20438,"scheduling_latency_s":0.204403}2179vm-test-run-nixbot-gitlab> gitlab # [ 226.162889] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.961Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"a9c4b0e7bdbbc2d258eb92af","created_at":"2026-06-30T07:13:22.683Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.685Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-a9c4b0e7bdbbc2d258eb92af: done: 0.071026 sec","job_status":"done","queue_duration_s":0.20438,"scheduling_latency_s":0.204403,"redis_calls":7,"redis_duration_s":0.003176,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000791,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.043,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.071026,"completed_at":"2026-06-30T07:13:22.961Z","load_balancing_strategy":"primary","db_duration_s":0.043329,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180vm-test-run-nixbot-gitlab> gitlab # [ 226.277534] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.963Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"78b91f92e9aeeda592042ca2","created_at":"2026-06-30T07:13:22.692Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.694Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-78b91f92e9aeeda592042ca2: start","job_status":"start","queue_duration_s":0.269409,"scheduling_latency_s":0.269445}2181vm-test-run-nixbot-gitlab> gitlab # [ 226.279080] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.996Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"78b91f92e9aeeda592042ca2","created_at":"2026-06-30T07:13:22.692Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.694Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-78b91f92e9aeeda592042ca2: done: 0.032918 sec","job_status":"done","queue_duration_s":0.269409,"scheduling_latency_s":0.269445,"redis_calls":10,"redis_duration_s":0.00347,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001076,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032918,"completed_at":"2026-06-30T07:13:22.996Z","load_balancing_strategy":"primary","db_duration_s":0.006839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2182vm-test-run-nixbot-gitlab> gitlab # [ 226.366439] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:22.998Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"74440f6d451327d495b0a66c","created_at":"2026-06-30T07:13:22.694Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.695Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-74440f6d451327d495b0a66c: start","job_status":"start","queue_duration_s":0.302489,"scheduling_latency_s":0.302514}2183vm-test-run-nixbot-gitlab> gitlab # [ 226.368267] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.022Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"74440f6d451327d495b0a66c","created_at":"2026-06-30T07:13:22.694Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.695Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-74440f6d451327d495b0a66c: done: 0.024018 sec","job_status":"done","queue_duration_s":0.302489,"scheduling_latency_s":0.302514,"redis_calls":7,"redis_duration_s":0.0023550000000000003,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000652,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024018,"completed_at":"2026-06-30T07:13:23.022Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184vm-test-run-nixbot-gitlab> gitlab # [ 226.457642] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.023Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"716ad82806c45897b1d8ee7d","created_at":"2026-06-30T07:13:22.696Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.698Z","job_size_bytes":19,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-716ad82806c45897b1d8ee7d: start","job_status":"start","queue_duration_s":0.325562,"scheduling_latency_s":0.325587}2185vm-test-run-nixbot-gitlab> gitlab # [ 226.459547] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.061Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"716ad82806c45897b1d8ee7d","created_at":"2026-06-30T07:13:22.696Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.698Z","job_size_bytes":19,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-716ad82806c45897b1d8ee7d: done: 0.037758 sec","job_status":"done","queue_duration_s":0.325562,"scheduling_latency_s":0.325587,"redis_calls":8,"redis_duration_s":0.0022589999999999997,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001653,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000606,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037758,"completed_at":"2026-06-30T07:13:23.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2186vm-test-run-nixbot-gitlab> gitlab # [ 226.551780] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.063Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"29d499dfcb223deef7e622b1","created_at":"2026-06-30T07:13:22.699Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.700Z","job_size_bytes":66,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-29d499dfcb223deef7e622b1: start","job_status":"start","queue_duration_s":0.362687,"scheduling_latency_s":0.362713}2187vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2188vm-test-run-nixbot-gitlab> gitlab # [ 226.553244] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.080Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"29d499dfcb223deef7e622b1","created_at":"2026-06-30T07:13:22.699Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.700Z","job_size_bytes":66,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-29d499dfcb223deef7e622b1: done: 0.017626 sec","job_status":"done","queue_duration_s":0.362687,"scheduling_latency_s":0.362713,"redis_calls":7,"redis_duration_s":0.001598,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001183,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017626,"completed_at":"2026-06-30T07:13:23.080Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2189vm-test-run-nixbot-gitlab> gitlab # [ 226.622130] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"96e2354f223233ff0236b10e","created_at":"2026-06-30T07:13:22.701Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.702Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-96e2354f223233ff0236b10e: start","job_status":"start","queue_duration_s":0.379358,"scheduling_latency_s":0.37939}2190vm-test-run-nixbot-gitlab> gitlab # [ 226.623598] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.107Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"96e2354f223233ff0236b10e","created_at":"2026-06-30T07:13:22.701Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.702Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-96e2354f223233ff0236b10e: done: 0.02486 sec","job_status":"done","queue_duration_s":0.379358,"scheduling_latency_s":0.37939,"redis_calls":7,"redis_duration_s":0.001692,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001253,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02486,"completed_at":"2026-06-30T07:13:23.106Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003117,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2191vm-test-run-nixbot-gitlab> gitlab # [ 226.740919] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.108Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"e8c7aeeeafdbf061e9203a96","created_at":"2026-06-30T07:13:22.743Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.745Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e8c7aeeeafdbf061e9203a96: start","job_status":"start","queue_duration_s":0.362897,"scheduling_latency_s":0.362922}2192vm-test-run-nixbot-gitlab> gitlab # [ 226.742241] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.127Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"e8c7aeeeafdbf061e9203a96","created_at":"2026-06-30T07:13:22.743Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.745Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e8c7aeeeafdbf061e9203a96: done: 0.019094 sec","job_status":"done","queue_duration_s":0.362897,"scheduling_latency_s":0.362922,"redis_calls":7,"redis_duration_s":0.0017959999999999999,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019094,"completed_at":"2026-06-30T07:13:23.127Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2193vm-test-run-nixbot-gitlab> gitlab # [ 226.779281] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.128Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"4fcfc51e73692d66696d44ce","created_at":"2026-06-30T07:13:22.746Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.747Z","job_size_bytes":6,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-4fcfc51e73692d66696d44ce: start","job_status":"start","queue_duration_s":0.381536,"scheduling_latency_s":0.381558}2194vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.27 seconds)2195vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-30T07:13:17.066Z', 'started_at': None, 'finished_at': '2026-06-30T07:13:22.325Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-30T07:13:22.515Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2196vm-test-run-nixbot-gitlab> gitlab # [ 226.846999] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.147Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"4fcfc51e73692d66696d44ce","created_at":"2026-06-30T07:13:22.746Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.747Z","job_size_bytes":6,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-4fcfc51e73692d66696d44ce: done: 0.018333 sec","job_status":"done","queue_duration_s":0.381536,"scheduling_latency_s":0.381558,"redis_calls":7,"redis_duration_s":0.002133,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001649,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018333,"completed_at":"2026-06-30T07:13:23.147Z","load_balancing_strategy":"primary","db_duration_s":0.000676,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197vm-test-run-nixbot-gitlab> gitlab # [ 226.918335] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.149Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"60de1bb6f2996aedb12b3c25","created_at":"2026-06-30T07:13:22.747Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.751Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-60de1bb6f2996aedb12b3c25: start","job_status":"start","queue_duration_s":0.397694,"scheduling_latency_s":0.397726}2198vm-test-run-nixbot-gitlab> gitlab # [ 226.919915] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.178Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"60de1bb6f2996aedb12b3c25","created_at":"2026-06-30T07:13:22.747Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.751Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-60de1bb6f2996aedb12b3c25: done: 0.029322 sec","job_status":"done","queue_duration_s":0.397694,"scheduling_latency_s":0.397726,"redis_calls":10,"redis_duration_s":0.002319,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001394,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345574,"duration_s":0.029322,"completed_at":"2026-06-30T07:13:23.178Z","load_balancing_strategy":"primary","db_duration_s":0.004873,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199vm-test-run-nixbot-gitlab> gitlab # [ 226.961962] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.179Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"98e13b18d99612d71741b5fe","created_at":"2026-06-30T07:13:22.801Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.804Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-98e13b18d99612d71741b5fe: start","job_status":"start","queue_duration_s":0.375688,"scheduling_latency_s":0.37571}2200vm-test-run-nixbot-gitlab> gitlab # [ 226.987161] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.398Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"98e13b18d99612d71741b5fe","created_at":"2026-06-30T07:13:22.801Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-5402acebe1c44c1f","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:22.804Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-98e13b18d99612d71741b5fe: done: 0.219044 sec","job_status":"done","queue_duration_s":0.375688,"scheduling_latency_s":0.37571,"gitaly_calls":3,"gitaly_duration_s":0.102563,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.012082,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001508,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003542,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000613,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002195,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000387,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000961,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083125,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.193770717,"duration_s":0.219044,"completed_at":"2026-06-30T07:13:23.398Z","load_balancing_strategy":"primary","db_duration_s":0.027998,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2201vm-test-run-nixbot-gitlab> gitlab # [ 227.129429] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.400Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"edf8ea4617284fdf62ee6d7c","created_at":"2026-06-30T07:13:23.230Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-c9ca8e0b480e4011","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:23.232Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-edf8ea4617284fdf62ee6d7c: start","job_status":"start","queue_duration_s":0.168372,"scheduling_latency_s":0.168396}2202vm-test-run-nixbot-gitlab> gitlab # [ 227.130930] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:23.428Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"edf8ea4617284fdf62ee6d7c","created_at":"2026-06-30T07:13:23.230Z","trace_propagation_headers":{"sentry-trace":"69f2d8828bc44f31800276e8cdb6aabd-c9ca8e0b480e4011","baggage":"sentry-trace_id=69f2d8828bc44f31800276e8cdb6aabd,sentry-release=Unknown"},"correlation_id":"01KWBP06PTNFS7Y155RTR5XS3M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:23.232Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-edf8ea4617284fdf62ee6d7c: done: 0.027687 sec","job_status":"done","queue_duration_s":0.168372,"scheduling_latency_s":0.168396,"redis_calls":7,"redis_duration_s":0.002326,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001649,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000677,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027687,"completed_at":"2026-06-30T07:13:23.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002941,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203vm-test-run-nixbot-gitlab> gitlab # [ 227.201739] gitaly[1372]: time="2026-06-30T07:13:23.242Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-30T07:13:53.240" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:23.241" grpc.time_ms=0.895 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2204vm-test-run-nixbot-gitlab> gitlab # [ 227.203539] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 2272205vm-test-run-nixbot-gitlab> gitlab # [ 227.204367] gitaly[1372]: time="2026-06-30T07:13:23.266Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:53.264" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:23.264" grpc.time_ms=1.107 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2206vm-test-run-nixbot-gitlab> gitlab # [ 227.205267] gitaly[1372]: time="2026-06-30T07:13:23.382Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP06PTNFS7Y155RTR5XS3M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:53.288" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:23.288" grpc.time_ms=93.081 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2207vm-test-run-nixbot-gitlab> gitlab # [ 227.273159] gitaly[1372]: time="2026-06-30T07:13:23.815Z" level=info msg="finished unary call with code OK" catfile.duration_ms=106 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=106 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP084SJS5PCAA9BRPF9SE6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:53.707" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:23.708" grpc.time_ms=107.426 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2208vm-test-run-nixbot-gitlab> gitlab # [ 227.431710] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:24.363Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"ed73f8f2e34ffa299c5dc3c3","created_at":"2026-06-30T07:13:05.843Z","trace_propagation_headers":{"sentry-trace":"64944dfd78c14d589f9ca35f57ee2c95-a6e0f068202a4593","baggage":"sentry-trace_id=64944dfd78c14d589f9ca35f57ee2c95,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPK5TA2ZS6H59TPJA4NJ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:509e1afd55f724a00464acb0a7c061b30c3ffaaa6ab52ef90c1b5346093cbd52","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:24.359Z","retry_count":0,"job_size_bytes":451,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-ed73f8f2e34ffa299c5dc3c3: start","job_status":"start","queue_duration_s":0.003413,"scheduling_latency_s":0.003487}2209vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2210vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.13 seconds)2211vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-30T07:13:17.066Z', 'started_at': None, 'finished_at': '2026-06-30T07:13:22.325Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-30T07:13:22.515Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2212vm-test-run-nixbot-gitlab> gitlab # [ 227.566883] sidekiq-cluster[2141]: {"severity":"WARN","time":"2026-06-30T07:13:24.496Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"ed73f8f2e34ffa299c5dc3c3","created_at":"2026-06-30T07:13:05.843Z","trace_propagation_headers":{"sentry-trace":"64944dfd78c14d589f9ca35f57ee2c95-a6e0f068202a4593","baggage":"sentry-trace_id=64944dfd78c14d589f9ca35f57ee2c95,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBNZPK5TA2ZS6H59TPJA4NJ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:509e1afd55f724a00464acb0a7c061b30c3ffaaa6ab52ef90c1b5346093cbd52","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:24.359Z","retry_count":0,"job_size_bytes":451,"pid":2141,"sidekiq_tid":"m9z9","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-ed73f8f2e34ffa299c5dc3c3: fail: 0.13293 sec","job_status":"fail","queue_duration_s":0.003413,"scheduling_latency_s":0.003487,"redis_calls":7,"redis_duration_s":0.00468,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003497,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.13293,"completed_at":"2026-06-30T07:13:24.495Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.008343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2213vm-test-run-nixbot-gitlab> gitlab # [ 228.136735] gitaly[1372]: time="2026-06-30T07:13:24.928Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP09CM0E4986BNSV7VWW3J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:54.025" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:24.926" grpc.time_ms=1.431 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2214vm-test-run-nixbot-gitlab> gitlab # [ 228.156863] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 812215vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2216vm-test-run-nixbot-gitlab> gitlab # [ 229.153584] gitaly[1372]: time="2026-06-30T07:13:26.084Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0AG3YR4ASPYSBP4B3VHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:56.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:26.083" grpc.time_ms=1.354 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2217vm-test-run-nixbot-gitlab> gitlab # [ 229.204110] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 882218vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.17 seconds)2219vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-30T07:13:17.066Z', 'started_at': None, 'finished_at': '2026-06-30T07:13:22.325Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-30T07:13:22.515Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2220vm-test-run-nixbot-gitlab> gitlab # [ 229.603983] nix-daemon[3480]: accepted connection from pid 3701, user nixbot2221vm-test-run-nixbot-gitlab> gitlab # [ 229.705915] gitaly[1372]: time="2026-06-30T07:13:26.638Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:56.635" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:26.636" grpc.time_ms=1.24 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2222vm-test-run-nixbot-gitlab> gitlab # [ 229.824346] gitaly[1372]: time="2026-06-30T07:13:26.756Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-30T07:13:36.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:26.740" grpc.time_ms=15.343 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2223vm-test-run-nixbot-gitlab> gitlab # [ 229.868377] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:26.798Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4a7bde4f026d9c532a6a25d9","created_at":"2026-06-30T07:13:26.790Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-7b96ea1e84a74214","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:26.794Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4a7bde4f026d9c532a6a25d9: start","job_status":"start","queue_duration_s":0.003168,"scheduling_latency_s":0.00324}2224vm-test-run-nixbot-gitlab> gitlab # [ 229.876905] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f9f27455b0fb535a117810be3b433070f1df3310 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2342225vm-test-run-nixbot-gitlab> gitlab # [ 229.880648] nixbot[3192]: {"ts": "2026-06-30T07:13:26.812Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f9f27455b0fb535a117810be3b433070f1df3310 \"HTTP/1.1 201 Created\""}2226vm-test-run-nixbot-gitlab> gitlab # [ 230.037382] gitaly[1372]: time="2026-06-30T07:13:26.969Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:26.969" grpc.time_ms=0.551 pid=1372 span.kind=server system=grpc2227vm-test-run-nixbot-gitlab> gitlab # [ 230.063206] nix-daemon[3480]: accepted connection from pid 3723, user nixbot2228vm-test-run-nixbot-gitlab> gitlab # [ 230.107189] gitaly[1372]: time="2026-06-30T07:13:27.039Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:27.039" grpc.time_ms=0.555 pid=1372 span.kind=server system=grpc2229vm-test-run-nixbot-gitlab> gitlab # [ 230.149875] gitaly[1372]: time="2026-06-30T07:13:27.082Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-30T07:13:27.081" grpc.time_ms=0.336 pid=1372 span.kind=server system=grpc2230vm-test-run-nixbot-gitlab> gitlab # [ 230.170531] gitaly[1372]: time="2026-06-30T07:13:27.102Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:56.022" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:26.923" operations="[repack_refs]" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2231vm-test-run-nixbot-gitlab> gitlab # [ 230.185204] gitaly[1372]: time="2026-06-30T07:13:27.117Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 diskcache=7fa5a3b5-3c79-424f-a42e-34b2b4b7fe26 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:56.022" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:26.923" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2232vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses2233vm-test-run-nixbot-gitlab> gitlab # [ 230.186255] gitaly[1372]: time="2026-06-30T07:13:27.117Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=156 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7254 command.oublock=0 command.real_time_ms=177 command.spawn_token_fork_ms=1 command.system_time_ms=110 command.user_time_ms=46 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:56.022" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:26.923" grpc.time_ms=194.617 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2234vm-test-run-nixbot-gitlab> gitlab # [ 230.190455] gitaly[1372]: time="2026-06-30T07:13:27.122Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWBP0BGSXSRW29BZ20JW8WRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:27.113" grpc.time_ms=8.981 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc2235vm-test-run-nixbot-gitlab> gitlab # [ 230.192708] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:27 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 582236vm-test-run-nixbot-gitlab> gitlab # [ 230.250684] gitaly[1372]: time="2026-06-30T07:13:27.183Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-30T07:13:57.181" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:27.181" grpc.time_ms=1.188 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2237vm-test-run-nixbot-gitlab> gitlab # [ 230.251930] gitaly[1372]: time="2026-06-30T07:13:27.183Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:56.022" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:26.923" pid=1372 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-06-30 07:13:06.583523441 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2238vm-test-run-nixbot-gitlab> gitlab # [ 230.283388] gitaly[1372]: time="2026-06-30T07:13:27.215Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:57.214" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:27.214" grpc.time_ms=0.816 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2239vm-test-run-nixbot-gitlab> gitlab # [ 230.321143] gitaly[1372]: time="2026-06-30T07:13:27.253Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:56.022" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:26.923" pid=1372 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2240vm-test-run-nixbot-gitlab> gitlab # [ 230.321803] gitaly[1372]: time="2026-06-30T07:13:27.253Z" level=info msg="optimized repository" clean_stale_data_time=58.695169ms clean_work_trees_time="292.774µs" commit_graph_time="8.102µs" correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.66µs" pid=1372 prune_time="7.822µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.939µs" storage_name=default system=housekeeping total_time=59.322064ms2241vm-test-run-nixbot-gitlab> gitlab # [ 230.322875] gitaly[1372]: time="2026-06-30T07:13:27.254Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-30T07:13:56.022" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-30T07:13:26.923" pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2242vm-test-run-nixbot-gitlab> gitlab # [ 230.330112] gitaly[1372]: time="2026-06-30T07:13:27.262Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0B1EQZ1GY1CAQPX85QB9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:57.259" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:27.260" grpc.time_ms=1.196 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2243vm-test-run-nixbot-gitlab> gitlab # [ 230.340221] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.272Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4a7bde4f026d9c532a6a25d9","created_at":"2026-06-30T07:13:26.790Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-7b96ea1e84a74214","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:26.794Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4a7bde4f026d9c532a6a25d9: done: 0.47471 sec","job_status":"done","queue_duration_s":0.003168,"scheduling_latency_s":0.00324,"gitaly_calls":4,"gitaly_duration_s":0.219212,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.038939,"redis_read_bytes":173,"redis_write_bytes":20927,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002199,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004989,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.008628,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13380,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.0141,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.0005,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002706,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003618,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.012,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.015,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.152119,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.42884808,"duration_s":0.47471,"completed_at":"2026-06-30T07:13:27.272Z","load_balancing_strategy":"primary","db_duration_s":0.055001,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2244vm-test-run-nixbot-gitlab> gitlab # [ 230.456469] gitaly[1372]: time="2026-06-30T07:13:27.286Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0BMSAN65N9Y7GWTSYVD0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-30T07:13:57.285" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-30T07:13:27.285" grpc.time_ms=0.468 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2245vm-test-run-nixbot-gitlab> gitlab # [ 230.458122] gitaly[1372]: time="2026-06-30T07:13:27.337Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0BJS7BMXQPHP6TD7YZ0W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:27.253" pid=1372 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2246vm-test-run-nixbot-gitlab> gitlab # [ 230.461918] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 1332247vm-test-run-nixbot-gitlab> gitlab # [ 230.463285] gitlab-workhorse[1280]: time="2026-06-30T07:13:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672248vm-test-run-nixbot-gitlab> gitlab # [ 230.463775] gitlab-workhorse[1280]: gitlab 2001:db8:1::1 - - [2026/06/30:07:13:27 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 2092249vm-test-run-nixbot-gitlab> gitlab # [ 230.465057] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.274Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"6e9ceb1f0892333307c6c8e4","created_at":"2026-06-30T07:13:26.909Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:26.912Z","job_size_bytes":23,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-6e9ceb1f0892333307c6c8e4: start","job_status":"start","queue_duration_s":0.361224,"scheduling_latency_s":0.361247}2250vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f9f27455b0fb535a117810be3b433070f1df3310/statuses, in 0.34 seconds)2251vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-30T07:13:17.066Z', 'started_at': None, 'finished_at': '2026-06-30T07:13:22.325Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'f9f27455b0fb535a117810be3b433070f1df3310', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-06-30T07:13:22.515Z', 'started_at': None, 'finished_at': '2026-06-30T07:13:26.782Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2252vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 11.22 seconds)2253vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 266.47 seconds)2254vm-test-run-nixbot-gitlab> gitlab # [ 230.540049] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.306Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"6e9ceb1f0892333307c6c8e4","created_at":"2026-06-30T07:13:26.909Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:26.912Z","job_size_bytes":23,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-6e9ceb1f0892333307c6c8e4: done: 0.032016 sec","job_status":"done","queue_duration_s":0.361224,"scheduling_latency_s":0.361247,"redis_calls":7,"redis_duration_s":0.001927,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032016,"completed_at":"2026-06-30T07:13:27.306Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004751,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2255vm-test-run-nixbot-gitlab> gitlab # [ 230.542171] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.307Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9af802e8a42df9efd0bd03a7","created_at":"2026-06-30T07:13:27.133Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.137Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9af802e8a42df9efd0bd03a7: start","job_status":"start","queue_duration_s":0.169676,"scheduling_latency_s":0.169699}2256vm-test-run-nixbot-gitlab> gitlab # [ 230.637815] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.330Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9af802e8a42df9efd0bd03a7","created_at":"2026-06-30T07:13:27.133Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.137Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9af802e8a42df9efd0bd03a7: done: 0.023161 sec","job_status":"done","queue_duration_s":0.169676,"scheduling_latency_s":0.169699,"redis_calls":7,"redis_duration_s":0.0017159999999999999,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023161,"completed_at":"2026-06-30T07:13:27.330Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257vm-test-run-nixbot-gitlab> gitlab # [ 230.639302] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.332Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6e52f2482515acf1c5de1c0a","created_at":"2026-06-30T07:13:27.144Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.146Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6e52f2482515acf1c5de1c0a: start","job_status":"start","queue_duration_s":0.185987,"scheduling_latency_s":0.186011}2258vm-test-run-nixbot-gitlab> gitlab # [ 230.719437] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.387Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6e52f2482515acf1c5de1c0a","created_at":"2026-06-30T07:13:27.144Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.146Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6e52f2482515acf1c5de1c0a: done: 0.054666 sec","job_status":"done","queue_duration_s":0.185987,"scheduling_latency_s":0.186011,"redis_calls":7,"redis_duration_s":0.024662999999999997,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.024196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054666,"completed_at":"2026-06-30T07:13:27.387Z","load_balancing_strategy":"primary","db_duration_s":0.004643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2259vm-test-run-nixbot-gitlab> gitlab # [ 230.792224] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.392Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"f2ddae59fd07cd71fbbe91cd","created_at":"2026-06-30T07:13:27.156Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.157Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f2ddae59fd07cd71fbbe91cd: start","job_status":"start","queue_duration_s":0.234266,"scheduling_latency_s":0.234299}2260vm-test-run-nixbot-gitlab> gitlab # [ 230.793928] gitaly[1372]: time="2026-06-30T07:13:27.337Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=78 command.spawn_token_fork_ms=76 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0BJS7BMXQPHP6TD7YZ0W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:27.253" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc2261vm-test-run-nixbot-gitlab> gitlab # [ 230.795353] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.425Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"f2ddae59fd07cd71fbbe91cd","created_at":"2026-06-30T07:13:27.156Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.157Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f2ddae59fd07cd71fbbe91cd: done: 0.033518 sec","job_status":"done","queue_duration_s":0.234266,"scheduling_latency_s":0.234299,"redis_calls":10,"redis_duration_s":0.003121,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000662,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001193,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033518,"completed_at":"2026-06-30T07:13:27.425Z","load_balancing_strategy":"primary","db_duration_s":0.004451,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2262vm-test-run-nixbot-gitlab> gitlab # [ 230.846359] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.427Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b375d6c371cd6aa8e9401eb9","created_at":"2026-06-30T07:13:27.158Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.160Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b375d6c371cd6aa8e9401eb9: start","job_status":"start","queue_duration_s":0.2669,"scheduling_latency_s":0.266924}2263vm-test-run-nixbot-gitlab> gitlab # [ 230.930546] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.446Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b375d6c371cd6aa8e9401eb9","created_at":"2026-06-30T07:13:27.158Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.160Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b375d6c371cd6aa8e9401eb9: done: 0.019503 sec","job_status":"done","queue_duration_s":0.2669,"scheduling_latency_s":0.266924,"redis_calls":7,"redis_duration_s":0.001725,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001224,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000501,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019503,"completed_at":"2026-06-30T07:13:27.446Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2264vm-test-run-nixbot-gitlab> gitlab # [ 230.940119] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.448Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"53073023a2bcc9c6538acfec","created_at":"2026-06-30T07:13:27.160Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.164Z","job_size_bytes":19,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-53073023a2bcc9c6538acfec: start","job_status":"start","queue_duration_s":0.284036,"scheduling_latency_s":0.284059}2265vm-test-run-nixbot-gitlab> gitlab # [ 231.018440] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.469Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"53073023a2bcc9c6538acfec","created_at":"2026-06-30T07:13:27.160Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.164Z","job_size_bytes":19,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-53073023a2bcc9c6538acfec: done: 0.021069 sec","job_status":"done","queue_duration_s":0.284036,"scheduling_latency_s":0.284059,"redis_calls":8,"redis_duration_s":0.002184,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001732,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021069,"completed_at":"2026-06-30T07:13:27.469Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002385,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2266vm-test-run-nixbot-gitlab> gitlab # [ 231.090170] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.470Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"faeae8a38ec6f6ca7396c159","created_at":"2026-06-30T07:13:27.164Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.166Z","job_size_bytes":66,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-faeae8a38ec6f6ca7396c159: start","job_status":"start","queue_duration_s":0.303876,"scheduling_latency_s":0.3039}2267vm-test-run-nixbot-gitlab> gitlab # [ 231.091418] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.505Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"faeae8a38ec6f6ca7396c159","created_at":"2026-06-30T07:13:27.164Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.166Z","job_size_bytes":66,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-faeae8a38ec6f6ca7396c159: done: 0.034509 sec","job_status":"done","queue_duration_s":0.303876,"scheduling_latency_s":0.3039,"redis_calls":7,"redis_duration_s":0.011552,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010464,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034509,"completed_at":"2026-06-30T07:13:27.505Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2268vm-test-run-nixbot-gitlab> gitlab # [ 231.128323] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.506Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"937a835c5c092dd126907952","created_at":"2026-06-30T07:13:27.167Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.169Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-937a835c5c092dd126907952: start","job_status":"start","queue_duration_s":0.337567,"scheduling_latency_s":0.3376}2269vm-test-run-nixbot-gitlab> gitlab # [ 231.154262] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.568Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"937a835c5c092dd126907952","created_at":"2026-06-30T07:13:27.167Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.169Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-937a835c5c092dd126907952: done: 0.061172 sec","job_status":"done","queue_duration_s":0.337567,"scheduling_latency_s":0.3376,"redis_calls":7,"redis_duration_s":0.001869,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001391,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000478,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061172,"completed_at":"2026-06-30T07:13:27.567Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2270vm-test-run-nixbot-gitlab> gitlab # [ 231.259063] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.569Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"78dda4032ce7fdf0f5cd6352","created_at":"2026-06-30T07:13:27.219Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.222Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-78dda4032ce7fdf0f5cd6352: start","job_status":"start","queue_duration_s":0.34719,"scheduling_latency_s":0.347224}2271vm-test-run-nixbot-gitlab> gitlab # [ 231.260250] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.608Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"78dda4032ce7fdf0f5cd6352","created_at":"2026-06-30T07:13:27.219Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.222Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-78dda4032ce7fdf0f5cd6352: done: 0.038567 sec","job_status":"done","queue_duration_s":0.34719,"scheduling_latency_s":0.347224,"redis_calls":7,"redis_duration_s":0.001956,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001408,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000548,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038567,"completed_at":"2026-06-30T07:13:27.608Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2272vm-test-run-nixbot-gitlab> gitlab # [ 231.328458] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.609Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0c2dd371271a46d7d8557d9f","created_at":"2026-06-30T07:13:27.222Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.225Z","job_size_bytes":6,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0c2dd371271a46d7d8557d9f: start","job_status":"start","queue_duration_s":0.384033,"scheduling_latency_s":0.384092}2273vm-test-run-nixbot-gitlab> gitlab # [ 231.404209] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.623Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0c2dd371271a46d7d8557d9f","created_at":"2026-06-30T07:13:27.222Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.225Z","job_size_bytes":6,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0c2dd371271a46d7d8557d9f: done: 0.013347 sec","job_status":"done","queue_duration_s":0.384033,"scheduling_latency_s":0.384092,"redis_calls":7,"redis_duration_s":0.001685,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013347,"completed_at":"2026-06-30T07:13:27.622Z","load_balancing_strategy":"primary","db_duration_s":0.000396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2274vm-test-run-nixbot-gitlab> gitlab # [ 231.406095] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.624Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"a2103ad6781f3d690b1f49fe","created_at":"2026-06-30T07:13:27.227Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.228Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a2103ad6781f3d690b1f49fe: start","job_status":"start","queue_duration_s":0.395739,"scheduling_latency_s":0.39576}2275vm-test-run-nixbot-gitlab> gitlab # [ 231.428117] sidekiq-cluster[2141]: {"severity":"INFO","time":"2026-06-30T07:13:27.641Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"a2103ad6781f3d690b1f49fe","created_at":"2026-06-30T07:13:27.227Z","trace_propagation_headers":{"sentry-trace":"94caa3c878db44a185cf38c6fad4dda4-472e40edc55d49f3","baggage":"sentry-trace_id=94caa3c878db44a185cf38c6fad4dda4,sentry-release=Unknown"},"correlation_id":"01KWBP0B1EQZ1GY1CAQPX85QB9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-30T07:13:27.228Z","job_size_bytes":3,"pid":2141,"sidekiq_tid":"m7rh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a2103ad6781f3d690b1f49fe: done: 0.016885 sec","job_status":"done","queue_duration_s":0.395739,"scheduling_latency_s":0.39576,"redis_calls":7,"redis_duration_s":0.001632,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000483,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016885,"completed_at":"2026-06-30T07:13:27.641Z","load_balancing_strategy":"primary","db_duration_s":0.002016,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2276vm-test-run-nixbot-gitlab> gitlab # [ 231.495938] gitaly[1372]: time="2026-06-30T07:13:27.338Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=78 command.spawn_token_fork_ms=76 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWBP0BJS7BMXQPHP6TD7YZ0W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-30T07:13:27.253" grpc.time_ms=84.742 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1372 remote_ip="2001:db8:1::1" span.kind=server system=grpc2277vm-test-run-nixbot-gitlab> gitlab # [ 231.568845] nix-daemon[3480]: accepted connection from pid 3787, user nixbot2278vm-test-run-nixbot-gitlab> test script finished in 267.52s2279vm-test-run-nixbot-gitlab> cleanup2280vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2281vm-test-run-nixbot-gitlab> gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2282vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.38 seconds)22832284post-build step Upload coverage to codecov: ok2285Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab