1these 13 derivations will be built:2 /nix/store/f94sw8vyclm5pqs79lbd2i44v3hi70yw-nixbot-effects.drv3 /nix/store/gvyza8z5p9xiqgwxjgc7x88klpjsrabg-unit-nixbot.service.drv4 /nix/store/gl8z3sj4pl9n0gz6pnl1n2h0k9h2zl11-system-units.drv5 /nix/store/s7jyxn7505i69j4dsmgp8zc0mb2flf09-etc.drv6 /nix/store/wvbh01ady05558kr9zvahzjrd3nm4prr-activate.drv7 /nix/store/q6liaqyll00hbczbkaj4iqkx5s8q6fh7-nixos-system-gitlab-test.drv8 /nix/store/q9h7bjddkipk2civ3zlh5kym4vxcx6xp-closure-info.drv9 /nix/store/mszfiia9pf7cc5x09fiy5kjw7a54b633-closure-info.drv10 /nix/store/hsnkhiaq727qri40b8f8mw0gzrnjqjq9-run-nixos-vm.drv11 /nix/store/m1xr94ila5qd9gvvzkqdij7w96sx7faz-nixos-vm.drv12 /nix/store/h962n6s5l8srhb6hx0rqfgvjk4hf0v2s-driverConfiguration.json.drv13 /nix/store/spin7r5a1vgkxihlsm0a77wl819bz1ip-nixos-test-driver-nixbot-gitlab.drv14 /nix/store/dn2d0d9zkm9445fzm5mzfpraphkr519j-vm-test-run-nixbot-gitlab.drv15building '/nix/store/f94sw8vyclm5pqs79lbd2i44v3hi70yw-nixbot-effects.drv' on 'ssh-ng://nix@jamie'16building '/nix/store/f94sw8vyclm5pqs79lbd2i44v3hi70yw-nixbot-effects.drv'17nixbot-effects> Sourcing python-remove-tests-dir-hook18nixbot-effects> Sourcing python-catch-conflicts-hook.sh19nixbot-effects> Sourcing python-remove-bin-bytecode-hook.sh20nixbot-effects> Sourcing pypa-build-hook21nixbot-effects> Using pypaBuildPhase22nixbot-effects> Sourcing python-runtime-deps-check-hook23nixbot-effects> Using pythonRuntimeDepsCheckHook24nixbot-effects> Sourcing pypa-install-hook25nixbot-effects> Using pypaInstallPhase26nixbot-effects> Sourcing python-imports-check-hook.sh27nixbot-effects> Using pythonImportsCheckPhase28nixbot-effects> Sourcing python-namespaces-hook29nixbot-effects> Sourcing python-catch-conflicts-hook.sh30nixbot-effects> Running phase: unpackPhase31nixbot-effects> unpacking source archive /nix/store/k00dadfbc7qlmhc9khnzd3z20z7ij5bk-nixbot_effects32nixbot-effects> source root is nixbot_effects33nixbot-effects> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot_effects/pyproject.toml"34nixbot-effects> Running phase: patchPhase35nixbot-effects> Running phase: updateAutotoolsGnuConfigScriptsPhase36nixbot-effects> Running phase: configurePhase37nixbot-effects> no configure script, doing nothing38nixbot-effects> Running phase: buildPhase39nixbot-effects> Executing pypaBuildPhase40nixbot-effects> Creating a wheel...41nixbot-effects> pypa build flags: --no-isolation --outdir dist/ --wheel42nixbot-effects> * Getting build dependencies for wheel...43nixbot-effects> * Building wheel...44nixbot-effects> Successfully built nixbot_effects-0.0.1-py3-none-any.whl45nixbot-effects> Finished creating a wheel...46nixbot-effects> Finished executing pypaBuildPhase47nixbot-effects> Running phase: pythonRuntimeDepsCheckHook48nixbot-effects> Executing pythonRuntimeDepsCheck49nixbot-effects> Checking runtime dependencies for nixbot_effects-0.0.1-py3-none-any.whl50nixbot-effects> Finished executing pythonRuntimeDepsCheck51nixbot-effects> Running phase: installPhase52nixbot-effects> Executing pypaInstallPhase53nixbot-effects> Successfully installed nixbot_effects-0.0.1-py3-none-any.whl54nixbot-effects> Finished executing pypaInstallPhase55nixbot-effects> Running phase: pythonOutputDistPhase56nixbot-effects> Executing pythonOutputDistPhase57nixbot-effects> Finished executing pythonOutputDistPhase58nixbot-effects> Running phase: fixupPhase59nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/9rv4cdg81d7kswvlzjnzr4xvy56akq7p-nixbot-effects60nixbot-effects> checking for references to /build/ in /nix/store/9rv4cdg81d7kswvlzjnzr4xvy56akq7p-nixbot-effects...61nixbot-effects> patching script interpreter paths in /nix/store/9rv4cdg81d7kswvlzjnzr4xvy56akq7p-nixbot-effects62nixbot-effects> stripping (with command strip and flags -S -p) in /nix/store/9rv4cdg81d7kswvlzjnzr4xvy56akq7p-nixbot-effects/lib /nix/store/9rv4cdg81d7kswvlzjnzr4xvy56akq7p-nixbot-effects/bin63nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/ycpxylqvy35l5pv3qdkbpcrqi2qcp0jz-nixbot-effects-dist64nixbot-effects> checking for references to /build/ in /nix/store/ycpxylqvy35l5pv3qdkbpcrqi2qcp0jz-nixbot-effects-dist...65nixbot-effects> patching script interpreter paths in /nix/store/ycpxylqvy35l5pv3qdkbpcrqi2qcp0jz-nixbot-effects-dist66nixbot-effects> Rewriting #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13 to #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.1367nixbot-effects> wrapping `/nix/store/9rv4cdg81d7kswvlzjnzr4xvy56akq7p-nixbot-effects/bin/nixbot-effects'...68nixbot-effects> Executing pythonRemoveTestsDir69nixbot-effects> Finished executing pythonRemoveTestsDir70nixbot-effects> Running phase: installCheckPhase71nixbot-effects> no Makefile or custom installCheckPhase, doing nothing72nixbot-effects> Running phase: pythonCatchConflictsPhase73nixbot-effects> Running phase: pythonRemoveBinBytecodePhase74nixbot-effects> Running phase: pythonImportsCheckPhase75nixbot-effects> Executing pythonImportsCheckPhase76building '/nix/store/gvyza8z5p9xiqgwxjgc7x88klpjsrabg-unit-nixbot.service.drv'77building '/nix/store/gl8z3sj4pl9n0gz6pnl1n2h0k9h2zl11-system-units.drv'78building '/nix/store/s7jyxn7505i69j4dsmgp8zc0mb2flf09-etc.drv'79building '/nix/store/wvbh01ady05558kr9zvahzjrd3nm4prr-activate.drv'80building '/nix/store/q6liaqyll00hbczbkaj4iqkx5s8q6fh7-nixos-system-gitlab-test.drv'81nixos-system-gitlab-test> structuredAttrs is enabled82building '/nix/store/q9h7bjddkipk2civ3zlh5kym4vxcx6xp-closure-info.drv'83closure-info> structuredAttrs is enabled84building '/nix/store/mszfiia9pf7cc5x09fiy5kjw7a54b633-closure-info.drv'85closure-info> structuredAttrs is enabled86building '/nix/store/hsnkhiaq727qri40b8f8mw0gzrnjqjq9-run-nixos-vm.drv'87building '/nix/store/m1xr94ila5qd9gvvzkqdij7w96sx7faz-nixos-vm.drv'88building '/nix/store/h962n6s5l8srhb6hx0rqfgvjk4hf0v2s-driverConfiguration.json.drv'89driverConfiguration.json> structuredAttrs is enabled90building '/nix/store/spin7r5a1vgkxihlsm0a77wl819bz1ip-nixos-test-driver-nixbot-gitlab.drv'91nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)92nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck93nixos-test-driver-nixbot-gitlab> All checks passed!94nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)95nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint96nixos-test-driver-nixbot-gitlab> All checks passed!97building '/nix/store/dn2d0d9zkm9445fzm5mzfpraphkr519j-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie'98building '/nix/store/dn2d0d9zkm9445fzm5mzfpraphkr519j-vm-test-run-nixbot-gitlab.drv'99vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state100vm-test-run-nixbot-gitlab> start all VLans101vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)102vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds103vm-test-run-nixbot-gitlab> run the VM test script104vm-test-run-nixbot-gitlab> additionally exposed symbols:105vm-test-run-nixbot-gitlab> gitlab,106vm-test-run-nixbot-gitlab> vlan1,107vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh108vm-test-run-nixbot-gitlab> start all VMs109vm-test-run-nixbot-gitlab> gitlab: starting vm110vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...111vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)112vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.gr6s3bzTPA', fmt=raw size=1073741824113vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)114vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done115vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes116vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 55333001-6a17-4223-8a4f-79b9cfb3907a117vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:118vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376119vm-test-run-nixbot-gitlab> gitlab # 120vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done121vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done122vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done123vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done124vm-test-run-nixbot-gitlab> gitlab # 125vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.126vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...127vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.128vm-test-run-nixbot-gitlab> (finished: start all VMs, in 34.88 seconds)129vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service130vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting131vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132vm-test-run-nixbot-gitlab> gitlab # 133vm-test-run-nixbot-gitlab> gitlab # 134vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00135vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...136vm-test-run-nixbot-gitlab> gitlab # 137vm-test-run-nixbot-gitlab> gitlab # 138vm-test-run-nixbot-gitlab> gitlab # 139vm-test-run-nixbot-gitlab> gitlab # 140vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00141vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...142vm-test-run-nixbot-gitlab> gitlab # 143vm-test-run-nixbot-gitlab> gitlab # 144vm-test-run-nixbot-gitlab> gitlab # Booting from ROM...145vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026146vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgga6mdwfkgknp4v9gcvdak0dcf9g8c1-nixos-system-gitlab-test/init regInfo=/nix/store/ggdc3yw51ihxc3f85rammg2v9ad3a4d8-closure-info/registration console=ttyS0,115200n8 console=tty0147vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-provided physical RAM map:148vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable149vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved150vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved151vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable152vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved153vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved154vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved155vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable156vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved157vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NX (Execute Disable) protection: active158vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] APIC: Static calls initialized159vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SMBIOS 2.8 present.160vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014161vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: Memory slots populated: 1/1162vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Hypervisor detected: KVM163vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000164vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00165vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] kvm-clock: using sched offset of 503004457 cycles166vm-test-run-nixbot-gitlab> gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns167vm-test-run-nixbot-gitlab> gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor168vm-test-run-nixbot-gitlab> gitlab # [ 0.000805] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000169vm-test-run-nixbot-gitlab> gitlab # [ 0.000848] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs170vm-test-run-nixbot-gitlab> gitlab # [ 0.000851] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT171vm-test-run-nixbot-gitlab> gitlab # [ 0.000903] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000172vm-test-run-nixbot-gitlab> gitlab # [ 0.002753] found SMP MP-table at [mem 0x000f5470-0x000f547f]173vm-test-run-nixbot-gitlab> gitlab # [ 0.002764] Using GB pages for direct mapping174vm-test-run-nixbot-gitlab> gitlab # [ 0.002861] RAMDISK: [mem 0xbe4e0000-0xbffcffff]175vm-test-run-nixbot-gitlab> gitlab # [ 0.002869] ACPI: Early table checksum verification disabled176vm-test-run-nixbot-gitlab> gitlab # [ 0.002872] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )177vm-test-run-nixbot-gitlab> gitlab # [ 0.002876] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)178vm-test-run-nixbot-gitlab> gitlab # [ 0.002880] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)179vm-test-run-nixbot-gitlab> gitlab # [ 0.002887] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)180vm-test-run-nixbot-gitlab> gitlab # [ 0.002889] ACPI: FACS 0x00000000BFFE0000 000040181vm-test-run-nixbot-gitlab> gitlab # [ 0.002890] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)182vm-test-run-nixbot-gitlab> gitlab # [ 0.002892] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)183vm-test-run-nixbot-gitlab> gitlab # [ 0.002894] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)184vm-test-run-nixbot-gitlab> gitlab # [ 0.002895] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]185vm-test-run-nixbot-gitlab> gitlab # [ 0.002896] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]186vm-test-run-nixbot-gitlab> gitlab # [ 0.002897] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]187vm-test-run-nixbot-gitlab> gitlab # [ 0.002897] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]188vm-test-run-nixbot-gitlab> gitlab # [ 0.002898] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]189vm-test-run-nixbot-gitlab> gitlab # [ 0.002898] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]190vm-test-run-nixbot-gitlab> gitlab # [ 0.003389] No NUMA configuration found191vm-test-run-nixbot-gitlab> gitlab # [ 0.003391] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]192vm-test-run-nixbot-gitlab> gitlab # [ 0.003394] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]193vm-test-run-nixbot-gitlab> gitlab # [ 0.005643] Zone ranges:194vm-test-run-nixbot-gitlab> gitlab # [ 0.005645] DMA [mem 0x0000000000001000-0x0000000000ffffff]195vm-test-run-nixbot-gitlab> gitlab # [ 0.005646] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]196vm-test-run-nixbot-gitlab> gitlab # [ 0.005647] Normal [mem 0x0000000100000000-0x00000001bfffffff]197vm-test-run-nixbot-gitlab> gitlab # [ 0.005648] Device empty198vm-test-run-nixbot-gitlab> gitlab # [ 0.005649] Movable zone start for each node199vm-test-run-nixbot-gitlab> gitlab # [ 0.005650] Early memory node ranges200vm-test-run-nixbot-gitlab> gitlab # [ 0.005650] node 0: [mem 0x0000000000001000-0x000000000009efff]201vm-test-run-nixbot-gitlab> gitlab # [ 0.005651] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]202vm-test-run-nixbot-gitlab> gitlab # [ 0.005652] node 0: [mem 0x0000000100000000-0x00000001bfffffff]203vm-test-run-nixbot-gitlab> gitlab # [ 0.005653] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]204vm-test-run-nixbot-gitlab> gitlab # [ 0.005672] On node 0, zone DMA: 1 pages in unavailable ranges205vm-test-run-nixbot-gitlab> gitlab # [ 0.005940] On node 0, zone DMA: 97 pages in unavailable ranges206vm-test-run-nixbot-gitlab> gitlab # [ 0.111360] On node 0, zone Normal: 40 pages in unavailable ranges207vm-test-run-nixbot-gitlab> gitlab # [ 0.112416] ACPI: PM-Timer IO Port: 0x608208vm-test-run-nixbot-gitlab> gitlab # [ 0.112433] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])209vm-test-run-nixbot-gitlab> gitlab # [ 0.112470] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23210vm-test-run-nixbot-gitlab> gitlab # [ 0.112473] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)211vm-test-run-nixbot-gitlab> gitlab # [ 0.112474] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)212vm-test-run-nixbot-gitlab> gitlab # [ 0.112475] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)213vm-test-run-nixbot-gitlab> gitlab # [ 0.112477] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)214vm-test-run-nixbot-gitlab> gitlab # [ 0.112477] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)215vm-test-run-nixbot-gitlab> gitlab # [ 0.112480] ACPI: Using ACPI (MADT) for SMP configuration information216vm-test-run-nixbot-gitlab> gitlab # [ 0.112481] ACPI: HPET id: 0x8086a201 base: 0xfed00000217vm-test-run-nixbot-gitlab> gitlab # [ 0.112488] TSC deadline timer available218vm-test-run-nixbot-gitlab> gitlab # [ 0.112492] CPU topo: Max. logical packages: 1219vm-test-run-nixbot-gitlab> gitlab # [ 0.112493] CPU topo: Max. logical dies: 1220vm-test-run-nixbot-gitlab> gitlab # [ 0.112493] CPU topo: Max. dies per package: 1221vm-test-run-nixbot-gitlab> gitlab # [ 0.112497] CPU topo: Max. threads per core: 1222vm-test-run-nixbot-gitlab> gitlab # [ 0.112497] CPU topo: Num. cores per package: 4223vm-test-run-nixbot-gitlab> gitlab # [ 0.112498] CPU topo: Num. threads per package: 4224vm-test-run-nixbot-gitlab> gitlab # [ 0.112498] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs225vm-test-run-nixbot-gitlab> gitlab # [ 0.112524] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()226vm-test-run-nixbot-gitlab> gitlab # [ 0.112542] kvm-guest: KVM setup pv remote TLB flush227vm-test-run-nixbot-gitlab> gitlab # [ 0.112545] kvm-guest: setup PV sched yield228vm-test-run-nixbot-gitlab> gitlab # [ 0.112565] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]229vm-test-run-nixbot-gitlab> gitlab # [ 0.112567] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]230vm-test-run-nixbot-gitlab> gitlab # [ 0.112568] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]231vm-test-run-nixbot-gitlab> gitlab # [ 0.112569] [mem 0xc0000000-0xfeffbfff] available for PCI devices232vm-test-run-nixbot-gitlab> gitlab # [ 0.112571] Booting paravirtualized kernel on KVM233vm-test-run-nixbot-gitlab> gitlab # [ 0.112575] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns234vm-test-run-nixbot-gitlab> gitlab # [ 0.117023] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1235vm-test-run-nixbot-gitlab> gitlab # [ 0.119205] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288236vm-test-run-nixbot-gitlab> gitlab # [ 0.119256] kvm-guest: PV spinlocks enabled237vm-test-run-nixbot-gitlab> gitlab # [ 0.119258] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)238vm-test-run-nixbot-gitlab> gitlab # [ 0.119261] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgga6mdwfkgknp4v9gcvdak0dcf9g8c1-nixos-system-gitlab-test/init regInfo=/nix/store/ggdc3yw51ihxc3f85rammg2v9ad3a4d8-closure-info/registration console=ttyS0,115200n8 console=tty0239vm-test-run-nixbot-gitlab> gitlab # [ 0.119356] Unknown kernel command line parameters "regInfo=/nix/store/ggdc3yw51ihxc3f85rammg2v9ad3a4d8-closure-info/registration", will be passed to user space.240vm-test-run-nixbot-gitlab> gitlab # [ 0.119369] random: crng init done241vm-test-run-nixbot-gitlab> gitlab # [ 0.119369] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes242vm-test-run-nixbot-gitlab> gitlab # [ 0.128037] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)243vm-test-run-nixbot-gitlab> gitlab # [ 0.132297] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)244vm-test-run-nixbot-gitlab> gitlab # [ 0.132372] software IO TLB: area num 4.245vm-test-run-nixbot-gitlab> gitlab # [ 0.204280] Fallback order for Node 0: 0246vm-test-run-nixbot-gitlab> gitlab # [ 0.204288] Built 1 zonelists, mobility grouping on. Total pages: 1572726247vm-test-run-nixbot-gitlab> gitlab # [ 0.204290] Policy zone: Normal248vm-test-run-nixbot-gitlab> gitlab # [ 0.206726] mem auto-init: stack:all(zero), heap alloc:on, heap free:off249vm-test-run-nixbot-gitlab> gitlab # [ 0.215705] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1250vm-test-run-nixbot-gitlab> gitlab # [ 0.228841] allocated 12582912 bytes of page_ext251vm-test-run-nixbot-gitlab> gitlab # [ 0.238598] ftrace: allocating 48565 entries in 192 pages252vm-test-run-nixbot-gitlab> gitlab # [ 0.238601] ftrace: allocated 192 pages with 2 groups253vm-test-run-nixbot-gitlab> gitlab # [ 0.239417] Dynamic Preempt: lazy254vm-test-run-nixbot-gitlab> gitlab # [ 0.239579] rcu: Preemptible hierarchical RCU implementation.255vm-test-run-nixbot-gitlab> gitlab # [ 0.239580] rcu: RCU event tracing is enabled.256vm-test-run-nixbot-gitlab> gitlab # [ 0.239580] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.257vm-test-run-nixbot-gitlab> gitlab # [ 0.239582] Trampoline variant of Tasks RCU enabled.258vm-test-run-nixbot-gitlab> gitlab # [ 0.239582] Rude variant of Tasks RCU enabled.259vm-test-run-nixbot-gitlab> gitlab # [ 0.239583] Tracing variant of Tasks RCU enabled.260vm-test-run-nixbot-gitlab> gitlab # [ 0.239584] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.261vm-test-run-nixbot-gitlab> gitlab # [ 0.239585] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4262vm-test-run-nixbot-gitlab> gitlab # [ 0.239610] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.263vm-test-run-nixbot-gitlab> gitlab # [ 0.239611] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.264vm-test-run-nixbot-gitlab> gitlab # [ 0.239612] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.265vm-test-run-nixbot-gitlab> gitlab # [ 0.243899] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16266vm-test-run-nixbot-gitlab> gitlab # [ 0.244210] rcu: srcu_init: Setting srcu_struct sizes based on contention.267vm-test-run-nixbot-gitlab> gitlab # [ 0.244320] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)268vm-test-run-nixbot-gitlab> gitlab # [ 0.251616] Console: colour VGA+ 80x25269vm-test-run-nixbot-gitlab> gitlab # [ 0.251620] printk: legacy console [tty0] enabled270vm-test-run-nixbot-gitlab> gitlab # [ 0.295514] printk: legacy console [ttyS0] enabled271vm-test-run-nixbot-gitlab> gitlab # [ 0.494685] ACPI: Core revision 20250807272vm-test-run-nixbot-gitlab> gitlab # [ 0.496226] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns273vm-test-run-nixbot-gitlab> gitlab # [ 0.499128] APIC: Switch to symmetric I/O mode setup274vm-test-run-nixbot-gitlab> gitlab # [ 0.500893] x2apic enabled275vm-test-run-nixbot-gitlab> gitlab # [ 0.502132] APIC: Switched APIC routing to: physical x2apic276vm-test-run-nixbot-gitlab> gitlab # [ 0.503836] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()277vm-test-run-nixbot-gitlab> gitlab # [ 0.506052] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()278vm-test-run-nixbot-gitlab> gitlab # [ 0.508826] kvm-guest: setup PV IPIs279vm-test-run-nixbot-gitlab> gitlab # [ 0.511162] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1280vm-test-run-nixbot-gitlab> gitlab # [ 0.512993] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns281vm-test-run-nixbot-gitlab> gitlab # [ 0.516127] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)282vm-test-run-nixbot-gitlab> gitlab # [ 0.517241] x86/cpu: User Mode Instruction Prevention (UMIP) activated283vm-test-run-nixbot-gitlab> gitlab # [ 0.519286] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127284vm-test-run-nixbot-gitlab> gitlab # [ 0.521125] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0285vm-test-run-nixbot-gitlab> gitlab # [ 0.522130] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto286vm-test-run-nixbot-gitlab> gitlab # [ 0.524126] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl287vm-test-run-nixbot-gitlab> gitlab # [ 0.525126] Transient Scheduler Attacks: Mitigation: Clear CPU buffers288vm-test-run-nixbot-gitlab> gitlab # [ 0.527125] Spectre V2 : Mitigation: Enhanced / Automatic IBRS289vm-test-run-nixbot-gitlab> gitlab # [ 0.529125] Speculative Return Stack Overflow: Mitigation: Safe RET290vm-test-run-nixbot-gitlab> gitlab # [ 0.531126] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization291vm-test-run-nixbot-gitlab> gitlab # [ 0.533132] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier292vm-test-run-nixbot-gitlab> gitlab # [ 0.534126] active return thunk: srso_alias_return_thunk293vm-test-run-nixbot-gitlab> gitlab # [ 0.536155] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'294vm-test-run-nixbot-gitlab> gitlab # [ 0.537126] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'295vm-test-run-nixbot-gitlab> gitlab # [ 0.539126] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'296vm-test-run-nixbot-gitlab> gitlab # [ 0.540125] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'297vm-test-run-nixbot-gitlab> gitlab # [ 0.541125] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'298vm-test-run-nixbot-gitlab> gitlab # [ 0.542125] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'299vm-test-run-nixbot-gitlab> gitlab # [ 0.543126] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'300vm-test-run-nixbot-gitlab> gitlab # [ 0.544125] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'301vm-test-run-nixbot-gitlab> gitlab # [ 0.546125] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'302vm-test-run-nixbot-gitlab> gitlab # [ 0.548126] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256303vm-test-run-nixbot-gitlab> gitlab # [ 0.549125] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64304vm-test-run-nixbot-gitlab> gitlab # [ 0.550126] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512305vm-test-run-nixbot-gitlab> gitlab # [ 0.551126] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024306vm-test-run-nixbot-gitlab> gitlab # [ 0.552125] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8307vm-test-run-nixbot-gitlab> gitlab # [ 0.554125] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16308vm-test-run-nixbot-gitlab> gitlab # [ 0.556125] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24309vm-test-run-nixbot-gitlab> gitlab # [ 0.557125] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.310vm-test-run-nixbot-gitlab> gitlab # [ 0.586199] Freeing SMP alternatives memory: 44K311vm-test-run-nixbot-gitlab> gitlab # [ 0.587128] pid_max: default: 32768 minimum: 301312vm-test-run-nixbot-gitlab> gitlab # [ 0.589216] LSM: initializing lsm=capability,landlock,yama,bpf,ima313vm-test-run-nixbot-gitlab> gitlab # [ 0.591144] landlock: Up and running.314vm-test-run-nixbot-gitlab> gitlab # [ 0.592125] Yama: becoming mindful.315vm-test-run-nixbot-gitlab> gitlab # [ 0.593334] LSM support for eBPF active316vm-test-run-nixbot-gitlab> gitlab # [ 0.594328] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)317vm-test-run-nixbot-gitlab> gitlab # [ 0.596265] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)318vm-test-run-nixbot-gitlab> gitlab # [ 0.599096] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)319vm-test-run-nixbot-gitlab> gitlab # [ 0.599731] Performance Events: Fam17h+ core perfctr, AMD PMU driver.320vm-test-run-nixbot-gitlab> gitlab # [ 0.600130] ... version: 2321vm-test-run-nixbot-gitlab> gitlab # [ 0.601127] ... bit width: 48322vm-test-run-nixbot-gitlab> gitlab # [ 0.602141] ... generic counters: 6323vm-test-run-nixbot-gitlab> gitlab # [ 0.603127] ... generic bitmap: 000000000000003f324vm-test-run-nixbot-gitlab> gitlab # [ 0.604127] ... fixed-purpose counters: 0325vm-test-run-nixbot-gitlab> gitlab # [ 0.605127] ... fixed-purpose bitmap: 0000000000000000326vm-test-run-nixbot-gitlab> gitlab # [ 0.606127] ... value mask: 0000ffffffffffff327vm-test-run-nixbot-gitlab> gitlab # [ 0.607127] ... max period: 00007fffffffffff328vm-test-run-nixbot-gitlab> gitlab # [ 0.608127] ... global_ctrl mask: 000000000000003f329vm-test-run-nixbot-gitlab> gitlab # [ 0.609267] signal: max sigframe size: 3376330vm-test-run-nixbot-gitlab> gitlab # [ 0.610198] rcu: Hierarchical SRCU implementation.331vm-test-run-nixbot-gitlab> gitlab # [ 0.611132] rcu: Max phase no-delay instances is 400.332vm-test-run-nixbot-gitlab> gitlab # [ 0.612344] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level333vm-test-run-nixbot-gitlab> gitlab # [ 0.617607] smp: Bringing up secondary CPUs ...334vm-test-run-nixbot-gitlab> gitlab # [ 0.618546] smpboot: x86: Booting SMP configuration:335vm-test-run-nixbot-gitlab> gitlab # [ 0.619138] .... node #0, CPUs: #1 #2 #3336vm-test-run-nixbot-gitlab> gitlab # [ 0.623292] smp: Brought up 1 node, 4 CPUs337vm-test-run-nixbot-gitlab> gitlab # [ 0.625127] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)338vm-test-run-nixbot-gitlab> gitlab # [ 0.626551] Memory: 6013140K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258292K reserved, 0K cma-reserved)339vm-test-run-nixbot-gitlab> gitlab # [ 0.629128] devtmpfs: initialized340vm-test-run-nixbot-gitlab> gitlab # [ 0.630361] x86/mm: Memory block size: 128MB341vm-test-run-nixbot-gitlab> gitlab # [ 0.634140] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns342vm-test-run-nixbot-gitlab> gitlab # [ 0.635175] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)343vm-test-run-nixbot-gitlab> gitlab # [ 0.636258] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).344vm-test-run-nixbot-gitlab> gitlab # [ 0.637245] pinctrl core: initialized pinctrl subsystem345vm-test-run-nixbot-gitlab> gitlab # [ 0.638476] PM: RTC time: 21:53:33, date: 2026-06-29346vm-test-run-nixbot-gitlab> gitlab # [ 0.641972] NET: Registered PF_NETLINK/PF_ROUTE protocol family347vm-test-run-nixbot-gitlab> gitlab # [ 0.644595] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations348vm-test-run-nixbot-gitlab> gitlab # [ 0.645182] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations349vm-test-run-nixbot-gitlab> gitlab # [ 0.647182] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations350vm-test-run-nixbot-gitlab> gitlab # [ 0.648142] audit: initializing netlink subsys (disabled)351vm-test-run-nixbot-gitlab> gitlab # [ 0.649336] audit: type=2000 audit(1782770013.805:1): state=initialized audit_enabled=0 res=1352vm-test-run-nixbot-gitlab> gitlab # [ 0.649464] thermal_sys: Registered thermal governor 'fair_share'353vm-test-run-nixbot-gitlab> gitlab # [ 0.651135] thermal_sys: Registered thermal governor 'bang_bang'354vm-test-run-nixbot-gitlab> gitlab # [ 0.652128] thermal_sys: Registered thermal governor 'step_wise'355vm-test-run-nixbot-gitlab> gitlab # [ 0.653128] thermal_sys: Registered thermal governor 'user_space'356vm-test-run-nixbot-gitlab> gitlab # [ 0.654127] thermal_sys: Registered thermal governor 'power_allocator'357vm-test-run-nixbot-gitlab> gitlab # [ 0.655155] cpuidle: using governor menu358vm-test-run-nixbot-gitlab> gitlab # [ 0.658375] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5359vm-test-run-nixbot-gitlab> gitlab # [ 0.659438] PCI: Using configuration type 1 for base access360vm-test-run-nixbot-gitlab> gitlab # [ 0.660128] PCI: Using configuration type 1 for extended access361vm-test-run-nixbot-gitlab> gitlab # [ 0.661303] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.362vm-test-run-nixbot-gitlab> gitlab # [ 0.664200] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages363vm-test-run-nixbot-gitlab> gitlab # [ 0.665128] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page364vm-test-run-nixbot-gitlab> gitlab # [ 0.666128] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages365vm-test-run-nixbot-gitlab> gitlab # [ 0.667127] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page366vm-test-run-nixbot-gitlab> gitlab # [ 0.672207] ACPI: Added _OSI(Module Device)367vm-test-run-nixbot-gitlab> gitlab # [ 0.676137] ACPI: Added _OSI(Processor Device)368vm-test-run-nixbot-gitlab> gitlab # [ 0.679135] ACPI: Added _OSI(Processor Aggregator Device)369vm-test-run-nixbot-gitlab> gitlab # [ 0.688019] ACPI: 1 ACPI AML tables successfully acquired and loaded370vm-test-run-nixbot-gitlab> gitlab # [ 0.696450] ACPI: Interpreter enabled371vm-test-run-nixbot-gitlab> gitlab # [ 0.698037] ACPI: PM: (supports S0 S3 S4 S5)372vm-test-run-nixbot-gitlab> gitlab # [ 0.699128] ACPI: Using IOAPIC for interrupt routing373vm-test-run-nixbot-gitlab> gitlab # [ 0.700231] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug374vm-test-run-nixbot-gitlab> gitlab # [ 0.706153] PCI: Using E820 reservations for host bridge windows375vm-test-run-nixbot-gitlab> gitlab # [ 0.710924] ACPI: Enabled 2 GPEs in block 00 to 0F376vm-test-run-nixbot-gitlab> gitlab # [ 0.728022] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])377vm-test-run-nixbot-gitlab> gitlab # [ 0.732151] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]378vm-test-run-nixbot-gitlab> gitlab # [ 0.738237] acpiphp: Slot [3] registered379vm-test-run-nixbot-gitlab> gitlab # [ 0.741221] acpiphp: Slot [4] registered380vm-test-run-nixbot-gitlab> gitlab # [ 0.743229] acpiphp: Slot [5] registered381vm-test-run-nixbot-gitlab> gitlab # [ 0.745169] acpiphp: Slot [6] registered382vm-test-run-nixbot-gitlab> gitlab # [ 0.746181] acpiphp: Slot [7] registered383vm-test-run-nixbot-gitlab> gitlab # [ 0.747170] acpiphp: Slot [8] registered384vm-test-run-nixbot-gitlab> gitlab # [ 0.748170] acpiphp: Slot [9] registered385vm-test-run-nixbot-gitlab> gitlab # [ 0.750169] acpiphp: Slot [10] registered386vm-test-run-nixbot-gitlab> gitlab # [ 0.751184] acpiphp: Slot [11] registered387vm-test-run-nixbot-gitlab> gitlab # [ 0.752171] acpiphp: Slot [12] registered388vm-test-run-nixbot-gitlab> gitlab # [ 0.753168] acpiphp: Slot [13] registered389vm-test-run-nixbot-gitlab> gitlab # [ 0.755177] acpiphp: Slot [14] registered390vm-test-run-nixbot-gitlab> gitlab # [ 0.756161] acpiphp: Slot [15] registered391vm-test-run-nixbot-gitlab> gitlab # [ 0.757163] acpiphp: Slot [16] registered392vm-test-run-nixbot-gitlab> gitlab # [ 0.759161] acpiphp: Slot [17] registered393vm-test-run-nixbot-gitlab> gitlab # [ 0.760174] acpiphp: Slot [18] registered394vm-test-run-nixbot-gitlab> gitlab # [ 0.761163] acpiphp: Slot [19] registered395vm-test-run-nixbot-gitlab> gitlab # [ 0.762161] acpiphp: Slot [20] registered396vm-test-run-nixbot-gitlab> gitlab # [ 0.764161] acpiphp: Slot [21] registered397vm-test-run-nixbot-gitlab> gitlab # [ 0.765175] acpiphp: Slot [22] registered398vm-test-run-nixbot-gitlab> gitlab # [ 0.766173] acpiphp: Slot [23] registered399vm-test-run-nixbot-gitlab> gitlab # [ 0.767161] acpiphp: Slot [24] registered400vm-test-run-nixbot-gitlab> gitlab # [ 0.769162] acpiphp: Slot [25] registered401vm-test-run-nixbot-gitlab> gitlab # [ 0.770173] acpiphp: Slot [26] registered402vm-test-run-nixbot-gitlab> gitlab # [ 0.771499] acpiphp: Slot [27] registered403vm-test-run-nixbot-gitlab> gitlab # [ 0.775225] acpiphp: Slot [28] registered404vm-test-run-nixbot-gitlab> gitlab # [ 0.778220] acpiphp: Slot [29] registered405vm-test-run-nixbot-gitlab> gitlab # [ 0.781241] acpiphp: Slot [30] registered406vm-test-run-nixbot-gitlab> gitlab # [ 0.784262] acpiphp: Slot [31] registered407vm-test-run-nixbot-gitlab> gitlab # [ 0.787262] PCI host bridge to bus 0000:00408vm-test-run-nixbot-gitlab> gitlab # [ 0.790151] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]409vm-test-run-nixbot-gitlab> gitlab # [ 0.795138] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]410vm-test-run-nixbot-gitlab> gitlab # [ 0.800139] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]411vm-test-run-nixbot-gitlab> gitlab # [ 0.805137] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]412vm-test-run-nixbot-gitlab> gitlab # [ 0.810185] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]413vm-test-run-nixbot-gitlab> gitlab # [ 0.814171] pci_bus 0000:00: root bus resource [bus 00-ff]414vm-test-run-nixbot-gitlab> gitlab # [ 0.817639] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint415vm-test-run-nixbot-gitlab> gitlab # [ 0.821129] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint416vm-test-run-nixbot-gitlab> gitlab # [ 0.824042] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint417vm-test-run-nixbot-gitlab> gitlab # [ 0.830129] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]418vm-test-run-nixbot-gitlab> gitlab # [ 0.831194] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk419vm-test-run-nixbot-gitlab> gitlab # [ 0.833128] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk420vm-test-run-nixbot-gitlab> gitlab # [ 0.835134] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk421vm-test-run-nixbot-gitlab> gitlab # [ 0.837148] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk422vm-test-run-nixbot-gitlab> gitlab # [ 0.839849] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint423vm-test-run-nixbot-gitlab> gitlab # [ 0.850396] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]424vm-test-run-nixbot-gitlab> gitlab # [ 0.857633] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint425vm-test-run-nixbot-gitlab> gitlab # [ 0.864748] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI426vm-test-run-nixbot-gitlab> gitlab # [ 0.870198] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB427vm-test-run-nixbot-gitlab> gitlab # [ 0.875303] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x180 took 10742 usecs428vm-test-run-nixbot-gitlab> gitlab # [ 0.880875] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint429vm-test-run-nixbot-gitlab> gitlab # [ 0.896213] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]430vm-test-run-nixbot-gitlab> gitlab # [ 0.898157] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]431vm-test-run-nixbot-gitlab> gitlab # [ 0.900197] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]432vm-test-run-nixbot-gitlab> gitlab # [ 0.902464] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]433vm-test-run-nixbot-gitlab> gitlab # [ 0.916134] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 13671 usecs434vm-test-run-nixbot-gitlab> gitlab # [ 0.919691] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint435vm-test-run-nixbot-gitlab> gitlab # [ 0.926143] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]436vm-test-run-nixbot-gitlab> gitlab # [ 0.928143] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]437vm-test-run-nixbot-gitlab> gitlab # [ 0.930182] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]438vm-test-run-nixbot-gitlab> gitlab # [ 0.932143] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]439vm-test-run-nixbot-gitlab> gitlab # [ 0.936366] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint440vm-test-run-nixbot-gitlab> gitlab # [ 0.941143] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]441vm-test-run-nixbot-gitlab> gitlab # [ 0.943143] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]442vm-test-run-nixbot-gitlab> gitlab # [ 0.945191] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]443vm-test-run-nixbot-gitlab> gitlab # [ 0.949155] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint444vm-test-run-nixbot-gitlab> gitlab # [ 0.954143] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]445vm-test-run-nixbot-gitlab> gitlab # [ 0.956143] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]446vm-test-run-nixbot-gitlab> gitlab # [ 0.958181] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]447vm-test-run-nixbot-gitlab> gitlab # [ 0.962142] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint448vm-test-run-nixbot-gitlab> gitlab # [ 0.966142] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]449vm-test-run-nixbot-gitlab> gitlab # [ 0.968143] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]450vm-test-run-nixbot-gitlab> gitlab # [ 0.970183] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]451vm-test-run-nixbot-gitlab> gitlab # [ 0.974618] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint452vm-test-run-nixbot-gitlab> gitlab # [ 0.987662] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]453vm-test-run-nixbot-gitlab> gitlab # [ 0.991172] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]454vm-test-run-nixbot-gitlab> gitlab # [ 0.995271] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]455vm-test-run-nixbot-gitlab> gitlab # [ 1.004212] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint456vm-test-run-nixbot-gitlab> gitlab # [ 1.015145] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]457vm-test-run-nixbot-gitlab> gitlab # [ 1.016167] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]458vm-test-run-nixbot-gitlab> gitlab # [ 1.018182] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]459vm-test-run-nixbot-gitlab> gitlab # [ 1.022620] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint460vm-test-run-nixbot-gitlab> gitlab # [ 1.028143] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]461vm-test-run-nixbot-gitlab> gitlab # [ 1.029142] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]462vm-test-run-nixbot-gitlab> gitlab # [ 1.032183] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]463vm-test-run-nixbot-gitlab> gitlab # [ 1.034143] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]464vm-test-run-nixbot-gitlab> gitlab # [ 1.037930] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint465vm-test-run-nixbot-gitlab> gitlab # [ 1.043157] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]466vm-test-run-nixbot-gitlab> gitlab # [ 1.045183] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]467vm-test-run-nixbot-gitlab> gitlab # [ 1.049180] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint468vm-test-run-nixbot-gitlab> gitlab # [ 1.056145] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]469vm-test-run-nixbot-gitlab> gitlab # [ 1.057143] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]470vm-test-run-nixbot-gitlab> gitlab # [ 1.059182] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]471vm-test-run-nixbot-gitlab> gitlab # [ 1.063593] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint472vm-test-run-nixbot-gitlab> gitlab # [ 1.068143] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]473vm-test-run-nixbot-gitlab> gitlab # [ 1.070143] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]474vm-test-run-nixbot-gitlab> gitlab # [ 1.071181] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]475vm-test-run-nixbot-gitlab> gitlab # [ 1.081370] ACPI: PCI: Interrupt link LNKA configured for IRQ 10476vm-test-run-nixbot-gitlab> gitlab # [ 1.083364] ACPI: PCI: Interrupt link LNKB configured for IRQ 10477vm-test-run-nixbot-gitlab> gitlab # [ 1.085338] ACPI: PCI: Interrupt link LNKC configured for IRQ 11478vm-test-run-nixbot-gitlab> gitlab # [ 1.087331] ACPI: PCI: Interrupt link LNKD configured for IRQ 11479vm-test-run-nixbot-gitlab> gitlab # [ 1.089240] ACPI: PCI: Interrupt link LNKS configured for IRQ 9480vm-test-run-nixbot-gitlab> gitlab # [ 1.093260] iommu: Default domain type: Translated481vm-test-run-nixbot-gitlab> gitlab # [ 1.094128] iommu: DMA domain TLB invalidation policy: lazy mode482vm-test-run-nixbot-gitlab> gitlab # [ 1.096499] ACPI: bus type USB registered483vm-test-run-nixbot-gitlab> gitlab # [ 1.098187] usbcore: registered new interface driver usbfs484vm-test-run-nixbot-gitlab> gitlab # [ 1.100156] usbcore: registered new interface driver hub485vm-test-run-nixbot-gitlab> gitlab # [ 1.101170] usbcore: registered new device driver usb486vm-test-run-nixbot-gitlab> gitlab # [ 1.104923] NetLabel: Initializing487vm-test-run-nixbot-gitlab> gitlab # [ 1.105945] NetLabel: domain hash size = 128488vm-test-run-nixbot-gitlab> gitlab # [ 1.107127] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO489vm-test-run-nixbot-gitlab> gitlab # [ 1.109227] NetLabel: unlabeled traffic allowed by default490vm-test-run-nixbot-gitlab> gitlab # [ 1.110145] PCI: Using ACPI for IRQ routing491vm-test-run-nixbot-gitlab> gitlab # [ 1.112708] pci 0000:00:02.0: vgaarb: setting as boot VGA device492vm-test-run-nixbot-gitlab> gitlab # [ 1.113122] pci 0000:00:02.0: vgaarb: bridge control possible493vm-test-run-nixbot-gitlab> gitlab # [ 1.113122] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none494vm-test-run-nixbot-gitlab> gitlab # [ 1.118131] vgaarb: loaded495vm-test-run-nixbot-gitlab> gitlab # [ 1.119311] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0496vm-test-run-nixbot-gitlab> gitlab # [ 1.121128] hpet0: 3 comparators, 64-bit 100.000000 MHz counter497vm-test-run-nixbot-gitlab> gitlab # [ 1.125349] clocksource: Switched to clocksource kvm-clock498vm-test-run-nixbot-gitlab> gitlab # [ 1.127643] VFS: Disk quotas dquot_6.6.0499vm-test-run-nixbot-gitlab> gitlab # [ 1.128916] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)500vm-test-run-nixbot-gitlab> gitlab # [ 1.131383] pnp: PnP ACPI init501vm-test-run-nixbot-gitlab> gitlab # [ 1.133079] pnp: PnP ACPI: found 6 devices502vm-test-run-nixbot-gitlab> gitlab # [ 1.143278] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns503vm-test-run-nixbot-gitlab> gitlab # [ 1.146042] clocksource: Switched to clocksource acpi_pm504vm-test-run-nixbot-gitlab> gitlab # [ 1.147828] NET: Registered PF_INET protocol family505vm-test-run-nixbot-gitlab> gitlab # [ 1.150696] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)506vm-test-run-nixbot-gitlab> gitlab # [ 1.170200] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)507vm-test-run-nixbot-gitlab> gitlab # [ 1.172715] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)508vm-test-run-nixbot-gitlab> gitlab # [ 1.175127] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)509vm-test-run-nixbot-gitlab> gitlab # [ 1.179957] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)510vm-test-run-nixbot-gitlab> gitlab # [ 1.182226] TCP: Hash tables configured (established 65536 bind 65536)511vm-test-run-nixbot-gitlab> gitlab # [ 1.184513] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)512vm-test-run-nixbot-gitlab> gitlab # [ 1.187144] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)513vm-test-run-nixbot-gitlab> gitlab # [ 1.189417] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)514vm-test-run-nixbot-gitlab> gitlab # [ 1.191691] NET: Registered PF_UNIX/PF_LOCAL protocol family515vm-test-run-nixbot-gitlab> gitlab # [ 1.193445] NET: Registered PF_XDP protocol family516vm-test-run-nixbot-gitlab> gitlab # [ 1.194989] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]517vm-test-run-nixbot-gitlab> gitlab # [ 1.196809] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]518vm-test-run-nixbot-gitlab> gitlab # [ 1.198662] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]519vm-test-run-nixbot-gitlab> gitlab # [ 1.200690] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]520vm-test-run-nixbot-gitlab> gitlab # [ 1.202709] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]521vm-test-run-nixbot-gitlab> gitlab # [ 1.205041] pci 0000:00:01.0: PIIX3: Enabling Passive Release522vm-test-run-nixbot-gitlab> gitlab # [ 1.206780] pci 0000:00:00.0: Limiting direct PCI/PCI transfers523vm-test-run-nixbot-gitlab> gitlab # [ 1.210248] ACPI: \_SB_.LNKD: Enabled at IRQ 11524vm-test-run-nixbot-gitlab> gitlab # [ 1.213694] PCI: CLS 0 bytes, default 64525vm-test-run-nixbot-gitlab> gitlab # [ 1.215119] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)526vm-test-run-nixbot-gitlab> gitlab # [ 1.215608] Trying to unpack rootfs image as initramfs...527vm-test-run-nixbot-gitlab> gitlab # [ 1.217000] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)528vm-test-run-nixbot-gitlab> gitlab # [ 1.217152] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns529vm-test-run-nixbot-gitlab> gitlab # [ 1.244126] Initialise system trusted keyrings530vm-test-run-nixbot-gitlab> gitlab # [ 1.246130] workingset: timestamp_bits=40 max_order=21 bucket_order=0531vm-test-run-nixbot-gitlab> gitlab # [ 1.273117] Key type asymmetric registered532vm-test-run-nixbot-gitlab> gitlab # [ 1.276369] Asymmetric key parser 'x509' registered533vm-test-run-nixbot-gitlab> gitlab # [ 1.280411] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)534vm-test-run-nixbot-gitlab> gitlab # [ 1.286286] io scheduler mq-deadline registered535vm-test-run-nixbot-gitlab> gitlab # [ 1.289088] io scheduler kyber registered536vm-test-run-nixbot-gitlab> gitlab # [ 1.297255] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled537vm-test-run-nixbot-gitlab> gitlab # [ 1.299535] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A538vm-test-run-nixbot-gitlab> gitlab # [ 1.303669] Linux agpgart interface v0.103539vm-test-run-nixbot-gitlab> gitlab # [ 1.305069] ACPI: bus type drm_connector registered540vm-test-run-nixbot-gitlab> gitlab # [ 1.309079] usbcore: registered new interface driver usbserial_generic541vm-test-run-nixbot-gitlab> gitlab # [ 1.310984] usbserial: USB Serial support registered for generic542vm-test-run-nixbot-gitlab> gitlab # [ 1.312769] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.543vm-test-run-nixbot-gitlab> gitlab # [ 1.312769] Please enable it if your BIOS has the CPPC option.544vm-test-run-nixbot-gitlab> gitlab # [ 1.316715] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled545vm-test-run-nixbot-gitlab> gitlab # [ 1.319152] drop_monitor: Initializing network drop monitor service546vm-test-run-nixbot-gitlab> gitlab # [ 1.321152] NET: Registered PF_INET6 protocol family547vm-test-run-nixbot-gitlab> gitlab # [ 1.324079] Segment Routing with IPv6548vm-test-run-nixbot-gitlab> gitlab # [ 1.325271] In-situ OAM (IOAM) with IPv6549vm-test-run-nixbot-gitlab> gitlab # [ 1.328679] IPI shorthand broadcast: enabled550vm-test-run-nixbot-gitlab> gitlab # [ 1.340338] sched_clock: Marking stable (1080075453, 259903714)->(1418768708, -78789541)551vm-test-run-nixbot-gitlab> gitlab # [ 1.347806] registered taskstats version 1552vm-test-run-nixbot-gitlab> gitlab # [ 1.352175] Loading compiled-in X.509 certificates553vm-test-run-nixbot-gitlab> gitlab # [ 1.376110] Demotion targets for Node 0: null554vm-test-run-nixbot-gitlab> gitlab # [ 1.379453] Key type .fscrypt registered555vm-test-run-nixbot-gitlab> gitlab # [ 1.381995] Key type fscrypt-provisioning registered556vm-test-run-nixbot-gitlab> gitlab # [ 1.385267] ima: No TPM chip found, activating TPM-bypass!557vm-test-run-nixbot-gitlab> gitlab # [ 1.388458] ima: Allocated hash algorithm: sha1558vm-test-run-nixbot-gitlab> gitlab # [ 1.390860] ima: No architecture policies found559vm-test-run-nixbot-gitlab> gitlab # [ 1.392790] PM: Magic number: 14:747:905560vm-test-run-nixbot-gitlab> gitlab # [ 1.395120] RAS: Correctable Errors collector initialized.561vm-test-run-nixbot-gitlab> gitlab # [ 1.400583] clk: Disabling unused clocks562vm-test-run-nixbot-gitlab> gitlab # [ 1.401843] PM: genpd: Disabling unused power domains563vm-test-run-nixbot-gitlab> gitlab # [ 1.506670] Freeing initrd memory: 27584K564vm-test-run-nixbot-gitlab> gitlab # [ 1.515305] Freeing unused decrypted memory: 2028K565vm-test-run-nixbot-gitlab> gitlab # [ 1.523843] Freeing unused kernel image (initmem) memory: 3644K566vm-test-run-nixbot-gitlab> gitlab # [ 1.528504] Write protecting the kernel read-only data: 32768k567vm-test-run-nixbot-gitlab> gitlab # [ 1.535868] Freeing unused kernel image (text/rodata gap) memory: 1332K568vm-test-run-nixbot-gitlab> gitlab # [ 1.541897] Freeing unused kernel image (rodata/data gap) memory: 816K569vm-test-run-nixbot-gitlab> gitlab # [ 1.599185] x86/mm: Checked W+X mappings: passed, no W+X pages found.570vm-test-run-nixbot-gitlab> gitlab # [ 1.601213] Run /init as init process571vm-test-run-nixbot-gitlab> gitlab # [ 1.616511] systemd[1]: Inserted module 'autofs4'572vm-test-run-nixbot-gitlab> gitlab # [ 1.644725] fuse: init (API version 7.45)573vm-test-run-nixbot-gitlab> gitlab # [ 1.655474] ACPI: \_SB_.LNKC: Enabled at IRQ 10574vm-test-run-nixbot-gitlab> gitlab # [ 1.665791] ACPI: \_SB_.LNKA: Enabled at IRQ 10575vm-test-run-nixbot-gitlab> gitlab # [ 1.682537] ACPI: \_SB_.LNKB: Enabled at IRQ 11576vm-test-run-nixbot-gitlab> gitlab # [ 1.732790] systemd[1]: Successfully made /usr/ read-only.577vm-test-run-nixbot-gitlab> gitlab # [ 2.071171] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)578vm-test-run-nixbot-gitlab> gitlab # [ 2.092505] systemd[1]: Detected virtualization kvm.579vm-test-run-nixbot-gitlab> gitlab # [ 2.096439] systemd[1]: Detected architecture x86-64.580vm-test-run-nixbot-gitlab> gitlab # [ 2.100403] systemd[1]: Running in initrd.581vm-test-run-nixbot-gitlab> gitlab # [ 2.104780] systemd[1]: Initializing machine ID from random generator.582vm-test-run-nixbot-gitlab> gitlab # [ 2.109565] systemd[1]: Hostname set to <gitlab>.583vm-test-run-nixbot-gitlab> gitlab # [ 2.186363] systemd[1]: Queued start job for default target Initrd Default Target.584vm-test-run-nixbot-gitlab> gitlab # [ 2.369836] systemd[1]: Created slice Slice /system/modprobe.585vm-test-run-nixbot-gitlab> gitlab # [ 2.375118] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.586vm-test-run-nixbot-gitlab> gitlab # [ 2.381241] systemd[1]: Expecting device /dev/disk/by-label/nix-store...587vm-test-run-nixbot-gitlab> gitlab # [ 2.386376] systemd[1]: Expecting device /dev/disk/by-label/nixos...588vm-test-run-nixbot-gitlab> gitlab # [ 2.391379] systemd[1]: Reached target Path Units.589vm-test-run-nixbot-gitlab> gitlab # [ 2.395018] systemd[1]: Reached target Slice Units.590vm-test-run-nixbot-gitlab> gitlab # [ 2.398164] systemd[1]: Reached target Swaps.591vm-test-run-nixbot-gitlab> gitlab # [ 2.401097] systemd[1]: Reached target Timer Units.592vm-test-run-nixbot-gitlab> gitlab # [ 2.404486] systemd[1]: Listening on D-Bus System Message Bus Socket.593vm-test-run-nixbot-gitlab> gitlab # [ 2.407098] systemd[1]: Listening on Journal Socket (/dev/log).594vm-test-run-nixbot-gitlab> gitlab # [ 2.409156] systemd[1]: Listening on Journal Sockets.595vm-test-run-nixbot-gitlab> gitlab # [ 2.410958] systemd[1]: Listening on udev Control Socket.596vm-test-run-nixbot-gitlab> gitlab # [ 2.412792] systemd[1]: Listening on udev Kernel Socket.597vm-test-run-nixbot-gitlab> gitlab # [ 2.414535] systemd[1]: Reached target Socket Units.598vm-test-run-nixbot-gitlab> gitlab # [ 2.421190] systemd[1]: Starting Create List of Static Device Nodes...599vm-test-run-nixbot-gitlab> gitlab # [ 2.427609] systemd[1]: Starting Load Kernel Module 9pnet_virtio...600vm-test-run-nixbot-gitlab> gitlab # [ 2.440864] systemd[1]: Starting Load Kernel Module configfs...601vm-test-run-nixbot-gitlab> gitlab # [ 2.455650] systemd[1]: Starting Journal Service...602vm-test-run-nixbot-gitlab> gitlab # [ 2.469814] systemd[1]: Starting Load Kernel Modules...603vm-test-run-nixbot-gitlab> gitlab # [ 2.474604] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki604vm-test-run-nixbot-gitlab> gitlab # [ 2.512375] systemd[1]: Starting Coldplug All udev Devices...605vm-test-run-nixbot-gitlab> gitlab # [ 2.516288] netfs: FS-Cache loaded606vm-test-run-nixbot-gitlab> gitlab # [ 2.521076] systemd-journald[152]: Collecting audit messages is disabled.607vm-test-run-nixbot-gitlab> gitlab # [ 2.525589] 9pnet: Installing 9P2000 support608vm-test-run-nixbot-gitlab> gitlab # [ 2.529595] systemd[1]: Finished Create List of Static Device Nodes.609vm-test-run-nixbot-gitlab> gitlab # [ 2.538363] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.610vm-test-run-nixbot-gitlab> gitlab # [ 2.542789] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.611vm-test-run-nixbot-gitlab> gitlab # [ 2.543006] systemd[1]: Finished Load Kernel Module 9pnet_virtio.612vm-test-run-nixbot-gitlab> gitlab # [ 2.543614] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev613vm-test-run-nixbot-gitlab> gitlab # [ 2.546389] systemd[1]: modprobe@configfs.service: Deactivated successfully.614vm-test-run-nixbot-gitlab> gitlab # [ 2.554863] systemd[1]: Finished Load Kernel Module configfs.615vm-test-run-nixbot-gitlab> gitlab # [ 2.561508] systemd[1]: Finished Load Kernel Modules.616vm-test-run-nixbot-gitlab> gitlab # [ 2.564007] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config617vm-test-run-nixbot-gitlab> gitlab # [ 2.569641] systemd[1]: Starting Apply Kernel Variables...618vm-test-run-nixbot-gitlab> gitlab # [ 2.577097] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...619vm-test-run-nixbot-gitlab> gitlab # [ 2.588201] systemd[1]: Finished Apply Kernel Variables.620vm-test-run-nixbot-gitlab> gitlab # [ 2.592480] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.621vm-test-run-nixbot-gitlab> gitlab # [ 2.334163] systemd-modules-l[ 2.595109] systemd[1]: Started Journal Service.622vm-test-run-nixbot-gitlab> gitlab # oad[154]: Using 4 probe threads623vm-test-run-nixbot-gitlab> gitlab # [ 2.338122] systemd-modules-load[154]: Inserted module 'virtio_balloon'624vm-test-run-nixbot-gitlab> gitlab # [ 2.339961] systemd-modules-load[154]: Inserted module 'virtio_gpu'625vm-test-run-nixbot-gitlab> gitlab # [ 2.341784] systemd-modules-load[154]: Inserted module 'dm_mod'626vm-test-run-nixbot-gitlab> gitlab # [ 2.343490] systemd[1]: Starting Create Static Device Nodes in /dev...627vm-test-run-nixbot-gitlab> gitlab # [ 2.355782] systemd[1]: Finished Create Static Device Nodes in /dev.628vm-test-run-nixbot-gitlab> gitlab # [ 2.357899] systemd[1]: Reached target Preparation for Local File Systems.629vm-test-run-nixbot-gitlab> gitlab # [ 2.359570] systemd[1]: Reached target Local File Systems.630vm-test-run-nixbot-gitlab> gitlab # [ 2.360898] systemd[1]: Starting Create System Files and Directories...631vm-test-run-nixbot-gitlab> gitlab # [ 2.370226] systemd[1]: Starting Rule-based Manager for Device Events and Files...632vm-test-run-nixbot-gitlab> gitlab # [ 2.378088] systemd[1]: Finished Create System Files and Directories.633vm-test-run-nixbot-gitlab> gitlab # [ 2.389858] systemd-udevd[197]: Using default interface naming scheme 'v260'.634vm-test-run-nixbot-gitlab> gitlab # [ 2.406285] systemd[1]: Started Rule-based Manager for Device Events and Files.635vm-test-run-nixbot-gitlab> gitlab # [ 2.450780] systemd[1]: Finished Coldplug All udev Devices.636vm-test-run-nixbot-gitlab> gitlab # [ 2.452503] systemd[1]: Reached target System Initialization.637vm-test-run-nixbot-gitlab> gitlab # [ 2.454075] systemd[1]: Reached target Basic System.638vm-test-run-nixbot-gitlab> gitlab # [ 2.838877] uhci_hcd 0000:00:01.2: UHCI Host Controller639vm-test-run-nixbot-gitlab> gitlab # [ 2.840805] SCSI subsystem initialized640vm-test-run-nixbot-gitlab> gitlab # [ 2.841175] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1641vm-test-run-nixbot-gitlab> gitlab # [ 2.845134] uhci_hcd 0000:00:01.2: detected 2 ports642vm-test-run-nixbot-gitlab> gitlab # [ 2.846542] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140643vm-test-run-nixbot-gitlab> gitlab # [ 2.848223] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18644vm-test-run-nixbot-gitlab> gitlab # [ 2.851190] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1645vm-test-run-nixbot-gitlab> gitlab # [ 2.854222] usb usb1: Product: UHCI Host Controller646vm-test-run-nixbot-gitlab> gitlab # [ 2.854783] virtio_blk virtio4: 4/0/0 default/read/poll queues647vm-test-run-nixbot-gitlab> gitlab # [ 2.856279] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd648vm-test-run-nixbot-gitlab> gitlab # [ 2.858681] usb usb1: SerialNumber: 0000:00:01.2649vm-test-run-nixbot-gitlab> gitlab # [ 2.867733] hub 1-0:1.0: USB hub found650vm-test-run-nixbot-gitlab> gitlab # [ 2.870110] hub 1-0:1.0: 2 ports detected651vm-test-run-nixbot-gitlab> gitlab # [ 2.873382] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12652vm-test-run-nixbot-gitlab> gitlab # [ 2.877532] serio: i8042 KBD port at 0x60,0x64 irq 1653vm-test-run-nixbot-gitlab> gitlab # [ 2.879840] serio: i8042 AUX port at 0x60,0x64 irq 12654vm-test-run-nixbot-gitlab> gitlab # [ 2.880107] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)655vm-test-run-nixbot-gitlab> gitlab # [ 2.887936] virtio_blk virtio5: 4/0/0 default/read/poll queues656vm-test-run-nixbot-gitlab> gitlab # [ 2.893635] virtio_blk virtio5: [vdb] 11498064 512-byte logical blocks (5.89 GB/5.48 GiB)657vm-test-run-nixbot-gitlab> gitlab # [ 2.915755] scsi host0: ata_piix658vm-test-run-nixbot-gitlab> gitlab # [ 2.918113] scsi host1: ata_piix659vm-test-run-nixbot-gitlab> gitlab # [ 2.919191] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0660vm-test-run-nixbot-gitlab> gitlab # [ 2.921853] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0661vm-test-run-nixbot-gitlab> gitlab # [ 2.676102] (udev-worker)[223]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.662vm-test-run-nixbot-gitlab> gitlab # [ 2.679990] (udev-worker)[223]: Network interface NamePolicy= disabled on kernel command line.663vm-test-run-nixbot-gitlab> gitlab # [ 2.682641] (udev-worker)[221]: Network interface NamePolicy= disabled on kernel command line.664vm-test-run-nixbot-gitlab> gitlab # [ 2.685777] systemd[1]: Starting Virtual Console Setup...665vm-test-run-nixbot-gitlab> gitlab # [ 2.704756] systemd[1]: Found device /dev/disk/by-label/nixos.666vm-test-run-nixbot-gitlab> gitlab # [ 2.708443] systemd[1]: Found device /dev/disk/by-label/nix-store.667vm-test-run-nixbot-gitlab> gitlab # [ 2.710400] systemd[1]: Reached target Initrd Root Device.668vm-test-run-nixbot-gitlab> gitlab # [ 2.711771] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...669vm-test-run-nixbot-gitlab> gitlab # [ 2.716243] systemd-vconsole-setup[237]: Configuration of first virtual console was skipped, ignoring remaining ones.670vm-test-run-nixbot-gitlab> gitlab # [ 2.719952] systemd[1]: Finished Virtual Console Setup.671vm-test-run-nixbot-gitlab> gitlab # [ 2.736615] systemd-fsck[245]: nixos: clean, 12/65536 files, 13019/262144 blocks672vm-test-run-nixbot-gitlab> gitlab # [ 2.741876] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.673vm-test-run-nixbot-gitlab> gitlab # [ 3.081293] ata2: found unknown device (class 0)674vm-test-run-nixbot-gitlab> gitlab # [ 3.086186] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100675vm-test-run-nixbot-gitlab> gitlab # [ 3.090228] usb 1-1: new full-speed USB device number 2 using uhci_hcd676vm-test-run-nixbot-gitlab> gitlab # [ 3.098042] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5677vm-test-run-nixbot-gitlab> gitlab # [ 3.132499] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0678vm-test-run-nixbot-gitlab> gitlab # [ 3.174276] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray679vm-test-run-nixbot-gitlab> gitlab # [ 3.204452] cdrom: Uniform CD-ROM driver Revision: 3.20680vm-test-run-nixbot-gitlab> gitlab # [ 3.271780] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00681vm-test-run-nixbot-gitlab> gitlab # [ 3.277419] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10682vm-test-run-nixbot-gitlab> gitlab # [ 3.281794] usb 1-1: Product: QEMU USB Tablet683vm-test-run-nixbot-gitlab> gitlab # [ 3.284660] usb 1-1: Manufacturer: QEMU684vm-test-run-nixbot-gitlab> gitlab # [ 3.287287] usb 1-1: SerialNumber: 28754-0000:00:01.2-1685vm-test-run-nixbot-gitlab> gitlab # [ 3.322131] hid: raw HID events driver (C) Jiri Kosina686vm-test-run-nixbot-gitlab> gitlab # [ 3.331516] usbcore: registered new interface driver usbhid687vm-test-run-nixbot-gitlab> gitlab # [ 3.332934] usbhid: USB HID core driver688vm-test-run-nixbot-gitlab> gitlab # [ 3.338969] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2689vm-test-run-nixbot-gitlab> gitlab # [ 3.342240] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0690vm-test-run-nixbot-gitlab> gitlab # [ 3.271987] systemd[1]: Mounting /sysroot...691vm-test-run-nixbot-gitlab> gitlab # [ 3.713497] EXT4-fs (vda): mounted filesystem 55333001-6a17-4223-8a4f-79b9cfb3907a r/w with ordered data mode. Quota mode: none.692vm-test-run-nixbot-gitlab> gitlab # [ 3.458916] systemd[1]: Mounted /sysroot.693vm-test-run-nixbot-gitlab> gitlab # [ 3.461500] systemd[1]: Reached target Initrd Root File System.694vm-test-run-nixbot-gitlab> gitlab # [ 3.467187] systemd[1]: Mounting /sysroot/nix/.ro-store...695vm-test-run-nixbot-gitlab> gitlab # [ 3.471792] systemd[1]: Mounting /sysroot/nix/.rw-store...696vm-test-run-nixbot-gitlab> gitlab # [ 3.483412] systemd[1]: Mounting /sysroot/run...697vm-test-run-nixbot-gitlab> gitlab # [ 3.487566] systemd[1]: Mounting /sysroot/tmp/shared...698vm-test-run-nixbot-gitlab> gitlab # [ 3.492152] systemd[1]: Mounting /sysroot/tmp/xchg...699vm-test-run-nixbot-gitlab> gitlab # [ 3.753715] erofs (device vdb): mounted with root inode @ nid 128.700vm-test-run-nixbot-gitlab> gitlab # [ 3.497351] systemd[1]: Starting Mountpoints Configured in the Real Root...701vm-test-run-nixbot-gitlab> gitlab # [ 3.499968] systemd[1]: Mounted /sysroot/nix/.ro-store.702vm-test-run-nixbot-gitlab> gitlab # [ 3.501353] systemd[1]: Mounted /sysroot/nix/.rw-store.703vm-test-run-nixbot-gitlab> gitlab # [ 3.768698] 9p: Installing v9fs 9p2000 file system support704vm-test-run-nixbot-gitlab> gitlab # [ 3.510145] systemd[1]: Mounted /sysroot/run.705vm-test-run-nixbot-gitlab> gitlab # [ 3.511680] systemd-sysroot-fstab-check[273]: /sysroot should be mounted in the initrd, will request daemon-reload.706vm-test-run-nixbot-gitlab> gitlab # [ 3.516463] systemd[1]: Mounted /sysroot/tmp/xchg.707vm-test-run-nixbot-gitlab> gitlab # [ 3.517692] systemd[1]: Mounted /sysroot/tmp/shared.708vm-test-run-nixbot-gitlab> gitlab # [ 3.522599] systemd[1]: Starting rw-sysroot-nix-store.service...709vm-test-run-nixbot-gitlab> gitlab # [ 3.525227] systemd[1]: Reload requested from client PID 273 ('systemd-sysroot') (unit initrd-parse-etc.service)...710vm-test-run-nixbot-gitlab> gitlab # [ 3.527828] systemd[1]: Reloading...711vm-test-run-nixbot-gitlab> gitlab # [ 3.738944] systemd[1]: Reloading finished in 213 ms.712vm-test-run-nixbot-gitlab> gitlab # [ 3.798362] systemd-sysroot-fstab-check[273]: Requesting initrd-fs.target/start/replace...713vm-test-run-nixbot-gitlab> gitlab # [ 3.942264] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.714vm-test-run-nixbot-gitlab> gitlab # [ 3.948220] systemd[1]: Finished rw-sysroot-nix-store.service.715vm-test-run-nixbot-gitlab> gitlab # [ 3.959167] systemd-sysroot-fstab-check[273]: Requesting swap.target/start/replace...716vm-test-run-nixbot-gitlab> gitlab # [ 3.969631] systemd[1]: Starting rw-sysroot-nix-store.service...717vm-test-run-nixbot-gitlab> gitlab # [ 3.973382] systemd[1]: initrd-parse-etc.service: Deactivated successfully.718vm-test-run-nixbot-gitlab> gitlab # [ 3.978457] systemd[1]: Finished Mountpoints Configured in the Real Root.719vm-test-run-nixbot-gitlab> gitlab # [ 3.982397] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.720vm-test-run-nixbot-gitlab> gitlab # [ 4.000991] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.721vm-test-run-nixbot-gitlab> gitlab # [ 4.008723] systemd[1]: Finished rw-sysroot-nix-store.service.722vm-test-run-nixbot-gitlab> gitlab # [ 4.273646] systemd[1]: Mounting /sysroot/nix/store...723vm-test-run-nixbot-gitlab> gitlab # [ 4.335773] systemd[1]: Mounted /sysroot/nix/store.724vm-test-run-nixbot-gitlab> gitlab # [ 4.339463] systemd[1]: Reached target Initrd File Systems.725vm-test-run-nixbot-gitlab> gitlab # [ 4.347180] systemd[1]: Starting Find NixOS closure...726vm-test-run-nixbot-gitlab> gitlab # [ 4.352944] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...727vm-test-run-nixbot-gitlab> gitlab # [ 4.394190] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.728vm-test-run-nixbot-gitlab> gitlab # [ 4.399396] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.729vm-test-run-nixbot-gitlab> gitlab # [ 4.414954] systemd[1]: Finished Find NixOS closure.730vm-test-run-nixbot-gitlab> gitlab # [ 4.420710] systemd[1]: Reached target Initrd Default Target.731vm-test-run-nixbot-gitlab> gitlab # [ 4.428798] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...732vm-test-run-nixbot-gitlab> gitlab # [ 4.471168] systemd[1]: Stopped target Initrd Default Target.733vm-test-run-nixbot-gitlab> gitlab # [ 4.475179] systemd[1]: Stopped target Basic System.734vm-test-run-nixbot-gitlab> gitlab # [ 4.478660] systemd[1]: Stopped target Initrd Root Device.735vm-test-run-nixbot-gitlab> gitlab # [ 4.482438] systemd[1]: Stopped target Path Units.736vm-test-run-nixbot-gitlab> gitlab # [ 4.485747] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.737vm-test-run-nixbot-gitlab> gitlab # [ 4.490594] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.738vm-test-run-nixbot-gitlab> gitlab # [ 4.495646] systemd[1]: Stopped target Slice Units.739vm-test-run-nixbot-gitlab> gitlab # [ 4.499232] systemd[1]: Stopped target Socket Units.740vm-test-run-nixbot-gitlab> gitlab # [ 4.502733] systemd[1]: Stopped target System Initialization.741vm-test-run-nixbot-gitlab> gitlab # [ 4.506606] systemd[1]: Stopped target Swaps.742vm-test-run-nixbot-gitlab> gitlab # [ 4.509851] systemd[1]: Stopped target Timer Units.743vm-test-run-nixbot-gitlab> gitlab # [ 4.513289] systemd[1]: dbus.socket: Deactivated successfully.744vm-test-run-nixbot-gitlab> gitlab # [ 4.516753] systemd[1]: Closed D-Bus System Message Bus Socket.745vm-test-run-nixbot-gitlab> gitlab # [ 4.520151] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.746vm-test-run-nixbot-gitlab> gitlab # [ 4.523708] systemd[1]: Stopped Find NixOS closure.747vm-test-run-nixbot-gitlab> gitlab # [ 4.526191] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio748vm-test-run-nixbot-gitlab> gitlab # [ 4.531235] systemd[1]: Starting rw-sysroot-nix-store.service...749vm-test-run-nixbot-gitlab> gitlab # [ 4.534208] systemd[1]: systemd-sysctl.service: Deactivated successfully.750vm-test-run-nixbot-gitlab> gitlab # [ 4.537127] systemd[1]: Stopped Apply Kernel Variables.751vm-test-run-nixbot-gitlab> gitlab # [ 4.539168] systemd[1]: systemd-modules-load.service: Deactivated successfully.752vm-test-run-nixbot-gitlab> gitlab # [ 4.540910] systemd[1]: Stopped Load Kernel Modules.753vm-test-run-nixbot-gitlab> gitlab # [ 4.542230] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.754vm-test-run-nixbot-gitlab> gitlab # [ 4.544169] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.755vm-test-run-nixbot-gitlab> gitlab # [ 4.546293] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.756vm-test-run-nixbot-gitlab> gitlab # [ 4.547988] systemd[1]: Stopped Create System Files and Directories.757vm-test-run-nixbot-gitlab> gitlab # [ 4.553983] systemd[1]: Stopped target Local File Systems.758vm-test-run-nixbot-gitlab> gitlab # [ 4.555414] systemd[1]: Stopped target Preparation for Local File Systems.759vm-test-run-nixbot-gitlab> gitlab # [ 4.557072] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.760vm-test-run-nixbot-gitlab> gitlab # [ 4.558797] systemd[1]: Stopped Coldplug All udev Devices.761vm-test-run-nixbot-gitlab> gitlab # [ 4.561555] systemd[1]: Stopping Rule-based Manager for Device Events and Files...762vm-test-run-nixbot-gitlab> gitlab # [ 4.563479] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.763vm-test-run-nixbot-gitlab> gitlab # [ 4.565325] systemd[1]: Stopped Virtual Console Setup.764vm-test-run-nixbot-gitlab> gitlab # [ 4.572829] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.765vm-test-run-nixbot-gitlab> gitlab # [ 4.575248] systemd[1]: Finished rw-sysroot-nix-store.service.766vm-test-run-nixbot-gitlab> gitlab # [ 4.582865] systemd[1]: systemd-udevd.service: Deactivated successfully.767vm-test-run-nixbot-gitlab> gitlab # [ 4.586454] systemd[1]: Stopped Rule-based Manager for Device Events and Files.768vm-test-run-nixbot-gitlab> gitlab # [ 4.588683] systemd[1]: systemd-udevd.service: Consumed 1.066s CPU time over 2.213s wall clock time, 26.2M memory peak.769vm-test-run-nixbot-gitlab> gitlab # [ 4.594654] systemd[1]: initrd-cleanup.service: Deactivated successfully.770vm-test-run-nixbot-gitlab> gitlab # [ 4.600194] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.771vm-test-run-nixbot-gitlab> gitlab # [ 4.606200] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.772vm-test-run-nixbot-gitlab> gitlab # [ 4.608454] systemd[1]: Closed udev Control Socket.773vm-test-run-nixbot-gitlab> gitlab # [ 4.617181] systemd[1]: Starting Cleanup udev Database...774vm-test-run-nixbot-gitlab> gitlab # [ 4.619207] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.775vm-test-run-nixbot-gitlab> gitlab # [ 4.621527] systemd[1]: Stopped Create Static Device Nodes in /dev.776vm-test-run-nixbot-gitlab> gitlab # [ 4.623506] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.777vm-test-run-nixbot-gitlab> gitlab # [ 4.625895] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.778vm-test-run-nixbot-gitlab> gitlab # [ 4.628231] systemd[1]: kmod-static-nodes.service: Deactivated successfully.779vm-test-run-nixbot-gitlab> gitlab # [ 4.630460] systemd[1]: Stopped Create List of Static Device Nodes.780vm-test-run-nixbot-gitlab> gitlab # [ 4.652803] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.781vm-test-run-nixbot-gitlab> gitlab # [ 4.655580] systemd[1]: Finished Cleanup udev Database.782vm-test-run-nixbot-gitlab> gitlab # [ 4.658866] systemd[1]: Reached target Switch Root.783vm-test-run-nixbot-gitlab> gitlab # [ 4.662453] systemd[1]: Starting NixOS Activation...784vm-test-run-nixbot-gitlab> gitlab # [ 4.782963] initrd-nixos-activation-start[572]: booting system configuration /nix/store/kgga6mdwfkgknp4v9gcvdak0dcf9g8c1-nixos-system-gitlab-test785vm-test-run-nixbot-gitlab> gitlab # [ 4.836524] initrd-nixos-activation-start[572]: running activation script...786vm-test-run-nixbot-gitlab> gitlab # [ 5.083190] initrd-nixos-activation-start[595]: setting up /etc...787vm-test-run-nixbot-gitlab> gitlab # [ 5.210723] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.788vm-test-run-nixbot-gitlab> gitlab # [ 5.216438] systemd[1]: Finished NixOS Activation.789vm-test-run-nixbot-gitlab> gitlab # [ 5.233385] systemd[1]: Starting Switch Root...790vm-test-run-nixbot-gitlab> gitlab # [ 5.285117] systemd[1]: Switching root.791vm-test-run-nixbot-gitlab> gitlab # [ 6.002779] systemd-journald[152]: Received SIGTERM from PID 1 (systemd).792vm-test-run-nixbot-gitlab> gitlab # [ 6.174260] NET: Registered PF_VSOCK protocol family793vm-test-run-nixbot-gitlab> gitlab # [ 6.551635] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)794vm-test-run-nixbot-gitlab> gitlab # [ 6.568079] systemd[1]: Detected virtualization kvm.795vm-test-run-nixbot-gitlab> gitlab # [ 6.571130] systemd[1]: Detected architecture x86-64.796vm-test-run-nixbot-gitlab> gitlab # [ 6.574353] systemd[1]: Detected first boot.797vm-test-run-nixbot-gitlab> gitlab # [ 6.578413] systemd[1]: Initializing machine ID from random generator.798vm-test-run-nixbot-gitlab> gitlab # [ 6.719315] systemd[1]: bpf-restrict-fs: LSM BPF program attached799vm-test-run-nixbot-gitlab> gitlab # [ 6.785256] systemd[1]: Applying preset policy.800vm-test-run-nixbot-gitlab> gitlab # [ 6.949155] systemd[1]: Populated /etc with preset unit settings.801vm-test-run-nixbot-gitlab> gitlab # [ 7.064617] systemd[1]: initrd-switch-root.service: Deactivated successfully.802vm-test-run-nixbot-gitlab> gitlab # [ 7.067037] systemd[1]: Stopped initrd-switch-root.service.803vm-test-run-nixbot-gitlab> gitlab # [ 7.070502] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.804vm-test-run-nixbot-gitlab> gitlab # [ 7.073937] systemd[1]: Created slice Slice /system/getty.805vm-test-run-nixbot-gitlab> gitlab # [ 7.076116] systemd[1]: Created slice GitLab DevOps Platform Slice.806vm-test-run-nixbot-gitlab> gitlab # [ 7.078402] systemd[1]: Created slice User and Session Slice.807vm-test-run-nixbot-gitlab> gitlab # [ 7.080035] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.808vm-test-run-nixbot-gitlab> gitlab # [ 7.082100] systemd[1]: Started Forward Password Requests to Wall Directory Watch.809vm-test-run-nixbot-gitlab> gitlab # [ 7.083984] systemd[1]: Expecting device /dev/hvc0...810vm-test-run-nixbot-gitlab> gitlab # [ 7.085314] systemd[1]: Expecting device /dev/ttyS0...811vm-test-run-nixbot-gitlab> gitlab # [ 7.086664] systemd[1]: Reached target Local Encrypted Volumes.812vm-test-run-nixbot-gitlab> gitlab # [ 7.088205] systemd[1]: Stopped target initrd-fs.target.813vm-test-run-nixbot-gitlab> gitlab # [ 7.089583] systemd[1]: Stopped target initrd-root-fs.target.814vm-test-run-nixbot-gitlab> gitlab # [ 7.091060] systemd[1]: Stopped target initrd-switch-root.target.815vm-test-run-nixbot-gitlab> gitlab # [ 7.092610] systemd[1]: Reached target Virtual Machines and Containers.816vm-test-run-nixbot-gitlab> gitlab # [ 7.094342] systemd[1]: Reached target Path Units.817vm-test-run-nixbot-gitlab> gitlab # [ 7.095619] systemd[1]: Reached target Remote File Systems.818vm-test-run-nixbot-gitlab> gitlab # [ 7.097117] systemd[1]: Reached target Slice Units.819vm-test-run-nixbot-gitlab> gitlab # [ 7.098441] systemd[1]: Reached target Swaps.820vm-test-run-nixbot-gitlab> gitlab # [ 7.100759] systemd[1]: Listening on Process Core Dump Socket.821vm-test-run-nixbot-gitlab> gitlab # [ 7.103158] systemd[1]: Listening on Credential Encryption/Decryption.822vm-test-run-nixbot-gitlab> gitlab # [ 7.106624] systemd[1]: Starting Journal Log Access Socket...823vm-test-run-nixbot-gitlab> gitlab # [ 7.108874] systemd[1]: Listening on Journal Audit Socket.824vm-test-run-nixbot-gitlab> gitlab # [ 7.110603] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.825vm-test-run-nixbot-gitlab> gitlab # [ 7.112516] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki826vm-test-run-nixbot-gitlab> gitlab # [ 7.115004] systemd[1]: Listening on udev Control Socket.827vm-test-run-nixbot-gitlab> gitlab # [ 7.118784] systemd[1]: Mounting Huge Pages File System...828vm-test-run-nixbot-gitlab> gitlab # [ 7.121667] systemd[1]: Mounting POSIX Message Queue File System...829vm-test-run-nixbot-gitlab> gitlab # [ 7.124791] systemd[1]: Mounting Kernel Debug File System...830vm-test-run-nixbot-gitlab> gitlab # [ 7.129031] systemd[1]: Mounting Kernel Trace File System...831vm-test-run-nixbot-gitlab> gitlab # [ 7.136861] systemd[1]: Starting Create List of Static Device Nodes...832vm-test-run-nixbot-gitlab> gitlab # [ 7.138575] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio833vm-test-run-nixbot-gitlab> gitlab # [ 7.143758] systemd[1]: Starting Load Kernel Module configfs...834vm-test-run-nixbot-gitlab> gitlab # [ 7.146473] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm835vm-test-run-nixbot-gitlab> gitlab # [ 7.149684] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore836vm-test-run-nixbot-gitlab> gitlab # [ 7.153203] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse837vm-test-run-nixbot-gitlab> gitlab # [ 7.157722] systemd[1]: Mounting FUSE Control File System...838vm-test-run-nixbot-gitlab> gitlab # [ 7.159315] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67839vm-test-run-nixbot-gitlab> gitlab # [ 7.180937] systemd[1]: Starting Journal Service...840vm-test-run-nixbot-gitlab> gitlab # [ 7.184748] systemd[1]: Starting Load Kernel Modules...841vm-test-run-nixbot-gitlab> gitlab # [ 7.193204] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...842vm-test-run-nixbot-gitlab> gitlab # [ 7.196973] systemd[1]: Starting Remount Root and Kernel File Systems...843vm-test-run-nixbot-gitlab> gitlab # [ 7.198694] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki844vm-test-run-nixbot-gitlab> gitlab # [ 7.205034] systemd[1]: Starting Coldplug All udev Devices...845vm-test-run-nixbot-gitlab> gitlab # [ 7.209533] systemd[1]: Listening on Journal Log Access Socket.846vm-test-run-nixbot-gitlab> gitlab # [ 7.210147] systemd-journald[806]: Collecting audit messages is enabled.847vm-test-run-nixbot-gitlab> gitlab # [ 7.211191] systemd[1]: Mounted Huge Pages File System.848vm-test-run-nixbot-gitlab> gitlab # [ 7.214352] systemd[1]: Mounted POSIX Message Queue File System.849vm-test-run-nixbot-gitlab> gitlab # [ 7.216124] systemd[1]: Mounted Kernel Debug File System.850vm-test-run-nixbot-gitlab> gitlab # [ 7.219237] systemd[1]: Mounted Kernel Trace File System.851vm-test-run-nixbot-gitlab> gitlab # [ 7.221468] systemd[1]: Finished Create List of Static Device Nodes.852vm-test-run-nixbot-gitlab> gitlab # [ 7.223509] systemd[1]: modprobe@configfs.service: Deactivated successfully.853vm-test-run-nixbot-gitlab> gitlab # [ 7.224158] loop: module loaded854vm-test-run-nixbot-gitlab> gitlab # [ 7.224954] EXT4-fs (vda): re-mounted 55333001-6a17-4223-8a4f-79b9cfb3907a.855vm-test-run-nixbot-gitlab> gitlab # [ 7.225945] systemd[1]: Finished Load Kernel Module configfs.856vm-test-run-nixbot-gitlab> gitlab # [ 7.230570] systemd[1]: Mounted FUSE Control File System.857vm-test-run-nixbot-gitlab> gitlab # [ 7.233506] systemd[1]: Finished Remount Root and Kernel File Systems.858vm-test-run-nixbot-gitlab> gitlab # [ 6.974643] systemd[1]: Queued start job for default target Multi-User System.859vm-test-run-nixbot-gitlab> gitlab # [ 6.977385] systemd[1]: Unnecessary job was removed for /dev/vdb.860vm-test-run-nixbot-gitlab> gitlab # [ 7.237118] systemd[1]: Started Journal Service.861vm-test-run-nixbot-gitlab> gitlab # [ 6.979546] systemd[1]: systemd-journald.service: Deactivated successfully.862vm-test-run-nixbot-gitlab> gitlab # [ 6.981989] systemd-modules-load[807]: Using 3 probe threads863vm-test-run-nixbot-gitlab> gitlab # [ 6.983863] systemd-modules-load[807]: Inserted module 'tls'864vm-test-run-nixbot-gitlab> gitlab # [ 6.985703] systemd-modules-load[807]: Inserted module 'loop'865vm-test-run-nixbot-gitlab> gitlab # [ 6.987503] systemd[1]: Finished Load Kernel Modules.866vm-test-run-nixbot-gitlab> gitlab # [ 6.989395] systemd-oomd[808]: No swap; memory pressure usage will be degraded867vm-test-run-nixbot-gitlab> gitlab # [ 6.991974] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.868vm-test-run-nixbot-gitlab> gitlab # [ 7.001764] systemd[1]: Mounting Kernel Configuration File System...869vm-test-run-nixbot-gitlab> gitlab # [ 7.005127] systemd[1]: Starting Firewall...870vm-test-run-nixbot-gitlab> gitlab # [ 7.019987] systemd[1]: Starting Flush Journal to Persistent Storage...871vm-test-run-nixbot-gitlab> gitlab # [ 7.021841] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore872vm-test-run-nixbot-gitlab> gitlab # [ 7.025280] systemd[1]: Starting Load/Save OS Random Seed...873vm-test-run-nixbot-gitlab> gitlab # [ 7.028231] systemd[1]: Starting Apply Kernel Variables...874vm-test-run-nixbot-gitlab> gitlab # [ 7.031257] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...875vm-test-run-nixbot-gitlab> gitlab # [ 7.034668] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki876vm-test-run-nixbot-gitlab> gitlab # [ 7.037123] systemd[1]: Mounted Kernel Configuration File System.877vm-test-run-nixbot-gitlab> gitlab # [ 7.308084] systemd-journald[806]: Received client request to flush runtime journal.878vm-test-run-nixbot-gitlab> gitlab # [ 7.113940] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.879vm-test-run-nixbot-gitlab> gitlab # [ 7.116062] systemd[1]: Finished Load/Save OS Random Seed.880vm-test-run-nixbot-gitlab> gitlab # [ 7.117665] systemd[1]: Reached target First Boot Complete.881vm-test-run-nixbot-gitlab> gitlab # [ 7.119291] systemd[1]: Starting Create Static Device Nodes in /dev...882vm-test-run-nixbot-gitlab> gitlab # [ 7.120844] systemd[1]: Finished Apply Kernel Variables.883vm-test-run-nixbot-gitlab> gitlab # [ 7.122238] systemd[1]: Finished Create Static Device Nodes in /dev.884vm-test-run-nixbot-gitlab> gitlab # [ 7.124222] systemd[1]: Reached target Preparation for Local File Systems.885vm-test-run-nixbot-gitlab> gitlab # [ 7.125969] systemd[1]: Starting Rule-based Manager for Device Events and Files...886vm-test-run-nixbot-gitlab> gitlab # [ 7.128307] systemd[1]: Finished Flush Journal to Persistent Storage.887vm-test-run-nixbot-gitlab> gitlab # [ 7.142670] systemd-udevd[849]: Using default interface naming scheme 'v260'.888vm-test-run-nixbot-gitlab> gitlab # [ 7.158559] systemd[1]: Finished Coldplug All udev Devices.889vm-test-run-nixbot-gitlab> gitlab # [ 7.183948] systemd[1]: Started Rule-based Manager for Device Events and Files.890vm-test-run-nixbot-gitlab> gitlab # [ 7.292845] systemd[1]: Finished Firewall.891vm-test-run-nixbot-gitlab> gitlab # [ 7.310945] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse892vm-test-run-nixbot-gitlab> gitlab # [ 7.334396] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.893vm-test-run-nixbot-gitlab> gitlab # [ 7.351146] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.894vm-test-run-nixbot-gitlab> gitlab # [ 7.364399] (udev-worker)[949]: Network interface NamePolicy= disabled on kernel command line.895vm-test-run-nixbot-gitlab> gitlab # [ 7.389760] (udev-worker)[934]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.896vm-test-run-nixbot-gitlab> gitlab # [ 7.396286] (udev-worker)[934]: Network interface NamePolicy= disabled on kernel command line.897vm-test-run-nixbot-gitlab> gitlab # [ 7.415855] systemd[1]: Condition check resulted in Virtio network device being skipped.898vm-test-run-nixbot-gitlab> gitlab # [ 7.418243] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore899vm-test-run-nixbot-gitlab> gitlab # [ 7.422358] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67900vm-test-run-nixbot-gitlab> gitlab # [ 7.426464] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore901vm-test-run-nixbot-gitlab> gitlab # [ 7.429273] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki902vm-test-run-nixbot-gitlab> gitlab # [ 7.431585] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki903vm-test-run-nixbot-gitlab> gitlab # [ 7.699979] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3904vm-test-run-nixbot-gitlab> gitlab # [ 7.707192] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0905vm-test-run-nixbot-gitlab> gitlab # [ 7.709093] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD906vm-test-run-nixbot-gitlab> gitlab # [ 7.712536] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console907vm-test-run-nixbot-gitlab> gitlab # [ 7.728212] Console: switching to colour dummy device 80x25908vm-test-run-nixbot-gitlab> gitlab # [ 7.729509] [drm] Found bochs VGA, ID 0xb0c5.909vm-test-run-nixbot-gitlab> gitlab # [ 7.729511] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.910vm-test-run-nixbot-gitlab> gitlab # [ 7.751105] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic911vm-test-run-nixbot-gitlab> gitlab # [ 7.752964] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0912vm-test-run-nixbot-gitlab> gitlab # [ 7.754601] parport_pc 00:03: reported by Plug and Play ACPI913vm-test-run-nixbot-gitlab> gitlab # [ 7.755019] Floppy drive(s): fd0 is 2.88M AMI BIOS914vm-test-run-nixbot-gitlab> gitlab # [ 7.758142] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]915vm-test-run-nixbot-gitlab> gitlab # [ 7.761775] rtc_cmos 00:05: RTC can wake from S4916vm-test-run-nixbot-gitlab> gitlab # [ 7.764103] ACPI: button: Power Button [PWRF]917vm-test-run-nixbot-gitlab> gitlab # [ 7.765227] fbcon: bochs-drmdrmfb (fb0) is primary device918vm-test-run-nixbot-gitlab> gitlab # [ 7.766558] rtc_cmos 00:05: registered as rtc0919vm-test-run-nixbot-gitlab> gitlab # [ 7.766703] rtc_cmos 00:05: setting system clock to 2026-06-29T21:53:40 UTC (1782770020)920vm-test-run-nixbot-gitlab> gitlab # [ 7.767633] FDC 0 is a S82078B921vm-test-run-nixbot-gitlab> gitlab # [ 7.768355] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs922vm-test-run-nixbot-gitlab> gitlab # [ 7.776335] Console: switching to colour frame buffer device 160x50923vm-test-run-nixbot-gitlab> gitlab # [ 7.776547] systemd-journald[806]: Time jumped backwards, rotating.924vm-test-run-nixbot-gitlab> gitlab # [ 7.789496] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device925vm-test-run-nixbot-gitlab> gitlab # [ 7.791801] mousedev: PS/2 mouse device common for all mice926vm-test-run-nixbot-gitlab> gitlab # [ 7.821867] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6927vm-test-run-nixbot-gitlab> gitlab # [ 7.822989] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4928vm-test-run-nixbot-gitlab> gitlab # [ 7.825954] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5929vm-test-run-nixbot-gitlab> gitlab # [ 7.610675] systemd[1]: Starting Virtual Console Setup...930vm-test-run-nixbot-gitlab> gitlab # [ 7.625175] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.931vm-test-run-nixbot-gitlab> gitlab # [ 7.629510] systemd[1]: Stopped Virtual Console Setup.932vm-test-run-nixbot-gitlab> gitlab # [ 7.636164] systemd[1]: Starting Virtual Console Setup...933vm-test-run-nixbot-gitlab> gitlab # [ 7.650103] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.934vm-test-run-nixbot-gitlab> gitlab # [ 7.652288] systemd[1]: Stopped Virtual Console Setup.935vm-test-run-nixbot-gitlab> gitlab # [ 7.671259] systemd[1]: Starting Virtual Console Setup...936vm-test-run-nixbot-gitlab> gitlab # [ 7.961230] ppdev: user-space parallel port driver937vm-test-run-nixbot-gitlab> gitlab # [ 7.979540] kvm_amd: TSC scaling supported938vm-test-run-nixbot-gitlab> gitlab # [ 7.980684] kvm_amd: Nested Virtualization enabled939vm-test-run-nixbot-gitlab> gitlab # [ 7.981943] kvm_amd: Nested Paging enabled940vm-test-run-nixbot-gitlab> gitlab # [ 7.982861] kvm_amd: LBR virtualization supported941vm-test-run-nixbot-gitlab> gitlab # [ 7.984027] kvm_amd: Virtual VMLOAD VMSAVE supported942vm-test-run-nixbot-gitlab> gitlab # [ 7.985253] kvm_amd: Virtual GIF supported943vm-test-run-nixbot-gitlab> gitlab # [ 7.986319] kvm_amd: Virtual NMI enabled944vm-test-run-nixbot-gitlab> gitlab # [ 8.038101] EDAC MC: Ver: 3.0.0945vm-test-run-nixbot-gitlab> gitlab # [ 7.782777] systemd-vconsole-setup[980]: Configuration of first virtual console was skipped, ignoring remaining ones.946vm-test-run-nixbot-gitlab> gitlab # [ 7.786895] systemd[1]: Finished Virtual Console Setup.947vm-test-run-nixbot-gitlab> gitlab # [ 7.806815] systemd[1]: Mounting /run/wrappers...948vm-test-run-nixbot-gitlab> gitlab # [ 7.824772] systemd[1]: Mounted /run/wrappers.949vm-test-run-nixbot-gitlab> gitlab # [ 7.826369] systemd[1]: Reached target Local File Systems.950vm-test-run-nixbot-gitlab> gitlab # [ 7.827869] systemd[1]: Listening on Boot Loader Control Service Socket.951vm-test-run-nixbot-gitlab> gitlab # [ 7.829804] systemd[1]: Starting register-nix-paths.service...952vm-test-run-nixbot-gitlab> gitlab # [ 7.831492] systemd[1]: Starting Create SUID/SGID Wrappers...953vm-test-run-nixbot-gitlab> gitlab # [ 7.833261] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.954vm-test-run-nixbot-gitlab> gitlab # [ 7.835686] systemd[1]: Starting Save Transient machine-id to Disk...955vm-test-run-nixbot-gitlab> gitlab # [ 7.837437] systemd[1]: Starting Create System Files and Directories...956vm-test-run-nixbot-gitlab> gitlab # [ 7.870143] systemd[1]: Finished Create System Files and Directories.957vm-test-run-nixbot-gitlab> gitlab # [ 7.873423] systemd[1]: Starting Rebuild Journal Catalog...958vm-test-run-nixbot-gitlab> gitlab # [ 7.875150] systemd[1]: Starting Record System Boot/Shutdown in UTMP...959vm-test-run-nixbot-gitlab> gitlab # [ 7.883393] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.960vm-test-run-nixbot-gitlab> gitlab # [ 7.888128] systemd[1]: Finished Save Transient machine-id to Disk.961vm-test-run-nixbot-gitlab> gitlab # [ 7.898264] systemd[1]: Finished Record System Boot/Shutdown in UTMP.962vm-test-run-nixbot-gitlab> gitlab # [ 7.912839] systemd[1]: Finished Rebuild Journal Catalog.963vm-test-run-nixbot-gitlab> gitlab # [ 7.916514] systemd[1]: Starting Update is Completed...964vm-test-run-nixbot-gitlab> gitlab # [ 7.934732] systemd[1]: Finished Update is Completed.965vm-test-run-nixbot-gitlab> gitlab # [ 7.985838] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.966vm-test-run-nixbot-gitlab> gitlab # [ 7.988150] systemd[1]: Finished Create SUID/SGID Wrappers.967vm-test-run-nixbot-gitlab> gitlab # [ 7.989629] systemd[1]: suid-sgid-wrappers.service: Consumed 127ms CPU time over 155ms wall clock time, 4.6M memory peak, 1.1M read from disk.968vm-test-run-nixbot-gitlab> gitlab # [ 8.234284] systemd[1]: Finished register-nix-paths.service.969vm-test-run-nixbot-gitlab> gitlab # [ 8.238119] systemd[1]: Reached target System Initialization.970vm-test-run-nixbot-gitlab> gitlab # [ 8.241678] systemd[1]: Started Discard unused filesystem blocks once a week.971vm-test-run-nixbot-gitlab> gitlab # [ 8.245861] systemd[1]: Started logrotate.timer.972vm-test-run-nixbot-gitlab> gitlab # [ 8.248810] systemd[1]: Started Daily Cleanup of Temporary Directories.973vm-test-run-nixbot-gitlab> gitlab # [ 8.252844] systemd[1]: Reached target Timer Units.974vm-test-run-nixbot-gitlab> gitlab # [ 8.255982] systemd[1]: Listening on D-Bus System Message Bus Socket.975vm-test-run-nixbot-gitlab> gitlab # [ 8.259996] systemd[1]: Listening on Nix Daemon Socket.976vm-test-run-nixbot-gitlab> gitlab # [ 8.263570] systemd[1]: Starting nixbot CI service socket...977vm-test-run-nixbot-gitlab> gitlab # [ 8.267227] systemd[1]: Listening on Hostname Service Socket.978vm-test-run-nixbot-gitlab> gitlab # [ 8.270806] systemd[1]: Listening on nixbot CI service socket.979vm-test-run-nixbot-gitlab> gitlab # [ 8.274669] systemd[1]: Reached target Socket Units.980vm-test-run-nixbot-gitlab> gitlab # [ 8.277884] systemd[1]: Starting D-Bus System Message Bus...981vm-test-run-nixbot-gitlab> gitlab # [ 8.339716] dbus-broker-launch[1070]: Looking up NSS user entry for 'systemd-timesync'...982vm-test-run-nixbot-gitlab> gitlab # [ 8.345541] dbus-broker-launch[1070]: NSS returned no entry for 'systemd-timesync'983vm-test-run-nixbot-gitlab> gitlab # [ 8.350313] dbus-broker-launch[1070]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"984vm-test-run-nixbot-gitlab> gitlab # [ 8.359644] systemd[1]: Started D-Bus System Message Bus.985vm-test-run-nixbot-gitlab> gitlab # [ 8.363624] dbus-broker-launch[1070]: Ready986vm-test-run-nixbot-gitlab> gitlab # [ 8.366833] systemd[1]: Reached target Basic System.987vm-test-run-nixbot-gitlab> gitlab # [ 8.369954] systemd[1]: Started backdoor.service.988vm-test-run-nixbot-gitlab> gitlab # [ 8.373155] systemd[1]: Starting gitlab-config.service...989vm-test-run-nixbot-gitlab> gitlab # [ 8.377700] systemd[1]: Starting Import lastlog data into lastlog2 database...990vm-test-run-nixbot-gitlab> gitlab # [ 8.381448] systemd[1]: Starting Logrotate configuration check...991vm-test-run-nixbot-gitlab> gitlab # [ 8.384397] systemd[1]: Starting Name Service Cache Daemon (nsncd)...992vm-test-run-nixbot-gitlab> gitlab # [ 8.391675] systemd[1]: Starting Post-Boot Actions...993vm-test-run-nixbot-gitlab> gitlab # [ 8.397305] systemd[1]: Started Reset console on configuration changes.994vm-test-run-nixbot-gitlab> gitlab # [ 8.403204] systemd[1]: Starting resolvconf update...995vm-test-run-nixbot-gitlab> gitlab # [ 8.410110] logrotate[1075]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.996vm-test-run-nixbot-gitlab> gitlab # [ 8.413909] logrotate[1075]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf997vm-test-run-nixbot-gitlab> gitlab # connecting to host...998vm-test-run-nixbot-gitlab> gitlab # [ 8.418497] logrotate[1075]: note: 'monthly' overrides previously specified 'weekly'999vm-test-run-nixbot-gitlab> gitlab # [ 8.420493] logrotate[1075]: Reading state from file: /var/lib/logrotate.status1000vm-test-run-nixbot-gitlab> gitlab # [ 8.422311] logrotate[1075]: state file /var/lib/logrotate.status does not exist1001vm-test-run-nixbot-gitlab> gitlab # [ 8.424406] logrotate[1075]: Allocating hash table for state file, size 64 entries1002vm-test-run-nixbot-gitlab> gitlab # [ 8.426859] logrotate[1075]: Handling 4 logs1003vm-test-run-nixbot-gitlab> gitlab # [ 8.428164] logrotate[1075]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1004vm-test-run-nixbot-gitlab> gitlab # [ 8.433270] logrotate[1075]: considering log /var/log/btmp1005vm-test-run-nixbot-gitlab> gitlab # [ 8.434379] logrotate[1075]: Creating new state1006vm-test-run-nixbot-gitlab> gitlab # [ 8.436330] logrotate[1075]: Now: 2026-06-29 21:531007vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'1008vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell1009vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.28 seconds)1010vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.28 seconds)1011vm-test-run-nixbot-gitlab> gitlab # [ 8.437651] logrotate[1075]: Last rotated at 2026-06-29 21:001012vm-test-run-nixbot-gitlab> gitlab # [ 8.441632] logrotate[1075]: log does not need rotating (log has already been rotated)1013vm-test-run-nixbot-gitlab> gitlab # [ 8.443753] logrotate[1075]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1014vm-test-run-nixbot-gitlab> gitlab # [ 8.447607] logrotate[1075]: considering log /var/log/wtmp1015vm-test-run-nixbot-gitlab> gitlab # [ 8.448948] logrotate[1075]: Creating new state1016vm-test-run-nixbot-gitlab> gitlab # [ 8.450352] logrotate[1075]: Now: 2026-06-29 21:531017vm-test-run-nixbot-gitlab> gitlab # [ 8.451374] logrotate[1075]: Last rotated at 2026-06-29 21:001018vm-test-run-nixbot-gitlab> gitlab # [ 8.452743] logrotate[1075]: log does not need rotating (log has already been rotated)1019vm-test-run-nixbot-gitlab> gitlab # [ 8.454458] logrotate[1075]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed1020vm-test-run-nixbot-gitlab> gitlab # [ 8.455813] logrotate[1075]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1075)1021vm-test-run-nixbot-gitlab> gitlab # [ 8.456421] logrotate[1075]: considering log /var/gitlab/state/log/*.log1022vm-test-run-nixbot-gitlab> gitlab # [ 8.457261] logrotate[1075]: log /var/gitlab/state/log/*.log does not exist -- skipping1023vm-test-run-nixbot-gitlab> gitlab # [ 8.458217] logrotate[1075]: Creating new state1024vm-test-run-nixbot-gitlab> gitlab # [ 8.458989] logrotate[1075]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1075)1025vm-test-run-nixbot-gitlab> gitlab # [ 8.460263] logrotate[1075]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed1026vm-test-run-nixbot-gitlab> gitlab # [ 8.461088] logrotate[1075]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1075)1027vm-test-run-nixbot-gitlab> gitlab # [ 8.462806] logrotate[1075]: considering log /var/log/nginx/*.log1028vm-test-run-nixbot-gitlab> gitlab # [ 8.463764] logrotate[1075]: log /var/log/nginx/*.log does not exist -- skipping1029vm-test-run-nixbot-gitlab> gitlab # [ 8.464586] logrotate[1075]: Creating new state1030vm-test-run-nixbot-gitlab> gitlab # [ 8.466089] logrotate[1075]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1075)1031vm-test-run-nixbot-gitlab> gitlab # [ 8.467076] nsncd[1076]: Jun 29 21:53:41.427 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1032vm-test-run-nixbot-gitlab> gitlab # [ 8.468580] systemd[1]: Finished Logrotate configuration check.1033vm-test-run-nixbot-gitlab> gitlab # [ 8.470428] systemd[1]: Finished Post-Boot Actions.1034vm-test-run-nixbot-gitlab> gitlab # [ 8.471786] systemd[1]: Started Name Service Cache Daemon (nsncd).1035vm-test-run-nixbot-gitlab> gitlab # [ 8.473977] systemd[1]: Reached target Host and Network Name Lookups.1036vm-test-run-nixbot-gitlab> gitlab # [ 8.475535] systemd[1]: Reached target User and Group Name Lookups.1037vm-test-run-nixbot-gitlab> gitlab # [ 8.477439] systemd[1]: Starting User Login Management...1038vm-test-run-nixbot-gitlab> gitlab # [ 8.478664] systemd[1]: Finished Import lastlog data into lastlog2 database.1039vm-test-run-nixbot-gitlab> gitlab # [ 8.497990] systemd[1]: Stopped target Host and Network Name Lookups.1040vm-test-run-nixbot-gitlab> gitlab # [ 8.499327] systemd[1]: Stopping Host and Network Name Lookups...1041vm-test-run-nixbot-gitlab> gitlab # [ 8.500138] systemd[1]: Stopped target User and Group Name Lookups.1042vm-test-run-nixbot-gitlab> gitlab # [ 8.500948] systemd[1]: Stopping User and Group Name Lookups...1043vm-test-run-nixbot-gitlab> gitlab # [ 8.502121] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...1044vm-test-run-nixbot-gitlab> gitlab # [ 8.508703] systemd[1]: nscd.service: Deactivated successfully.1045vm-test-run-nixbot-gitlab> gitlab # [ 8.511270] systemd[1]: Stopped Name Service Cache Daemon (nsncd).1046vm-test-run-nixbot-gitlab> gitlab # [ 8.512377] systemd[1]: nscd.service: Consumed 28ms CPU time over 132ms wall clock time, 8M memory peak, 5.6M read from disk.1047vm-test-run-nixbot-gitlab> gitlab # [ 8.524401] systemd-logind[1111]: New seat seat0.1048vm-test-run-nixbot-gitlab> gitlab # [ 8.526333] systemd[1]: Starting Name Service Cache Daemon (nsncd)...1049vm-test-run-nixbot-gitlab> gitlab # [ 8.529148] systemd-logind[1111]: Watching system buttons on /dev/input/event2 (Power Button)1050vm-test-run-nixbot-gitlab> gitlab # [ 8.530414] systemd-logind[1111]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)1051vm-test-run-nixbot-gitlab> gitlab # [ 8.533358] systemd-logind[1111]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)1052vm-test-run-nixbot-gitlab> gitlab # [ 8.534196] systemd[1]: Started User Login Management.1053vm-test-run-nixbot-gitlab> gitlab # [ 8.545848] systemd[1]: Starting linger-users.service...1054vm-test-run-nixbot-gitlab> gitlab # [ 8.557199] systemd[1]: Finished resolvconf update.1055vm-test-run-nixbot-gitlab> gitlab # [ 8.558696] nsncd[1160]: Jun 29 21:53:41.551 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1056vm-test-run-nixbot-gitlab> gitlab # [ 8.559909] systemd[1]: Started Name Service Cache Daemon (nsncd).1057vm-test-run-nixbot-gitlab> gitlab # [ 8.561790] systemd[1]: linger-users.service: Deactivated successfully.1058vm-test-run-nixbot-gitlab> gitlab # [ 8.564171] systemd[1]: Finished linger-users.service.1059vm-test-run-nixbot-gitlab> gitlab # [ 8.565292] systemd[1]: Reached target Preparation for Network.1060vm-test-run-nixbot-gitlab> gitlab # [ 8.566405] systemd[1]: Reached target Host and Network Name Lookups.1061vm-test-run-nixbot-gitlab> gitlab # [ 8.567888] systemd[1]: Reached target User and Group Name Lookups.1062vm-test-run-nixbot-gitlab> gitlab # [ 8.569223] systemd[1]: Starting DHCP Client...1063vm-test-run-nixbot-gitlab> gitlab # [ 8.570431] systemd[1]: Starting Address configuration of eth1...1064vm-test-run-nixbot-gitlab> gitlab # [ 8.575077] systemd[1]: Starting Extra networking commands....1065vm-test-run-nixbot-gitlab> gitlab # [ 8.603710] network-addresses-eth1-start[1188]: adding address 192.168.1.1/24... done1066vm-test-run-nixbot-gitlab> gitlab # [ 8.609766] network-addresses-eth1-start[1188]: adding address 2001:db8:1::1/64... done1067vm-test-run-nixbot-gitlab> gitlab # [ 8.616212] systemd[1]: Finished Address configuration of eth1.1068vm-test-run-nixbot-gitlab> gitlab # [ 8.647173] systemd[1]: Finished Extra networking commands..1069vm-test-run-nixbot-gitlab> gitlab # [ 8.648747] systemd[1]: Reached target Network.1070vm-test-run-nixbot-gitlab> gitlab # [ 8.650663] systemd[1]: Starting gitlab-workhorse.service...1071vm-test-run-nixbot-gitlab> gitlab # [ 8.657609] dhcpcd[1203]: dhcpcd-10.3.2 starting1072vm-test-run-nixbot-gitlab> gitlab # [ 8.659711] systemd[1]: Starting Nginx Web Server...1073vm-test-run-nixbot-gitlab> gitlab # [ 8.660833] systemd[1]: Starting PostgreSQL Server...1074vm-test-run-nixbot-gitlab> gitlab # [ 8.662245] dhcpcd[1249]: dev: loaded udev1075vm-test-run-nixbot-gitlab> gitlab # [ 8.664471] systemd[1]: Starting Redis Server - redis-gitlab...1076vm-test-run-nixbot-gitlab> gitlab # [ 8.927273] 8021q: 802.1Q VLAN Support v1.81077vm-test-run-nixbot-gitlab> gitlab # [ 8.928391] 8021q: adding VLAN 0 to HW filter on device eth11078vm-test-run-nixbot-gitlab> gitlab # [ 8.670202] systemd[1]: Starting Permit User Sessions...1079vm-test-run-nixbot-gitlab> gitlab # [ 8.692909] systemd[1]: Finished Permit User Sessions.1080vm-test-run-nixbot-gitlab> gitlab # [ 8.695462] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1081vm-test-run-nixbot-gitlab> gitlab # [ 8.708211] systemd[1]: Started Getty on tty1.1082vm-test-run-nixbot-gitlab> gitlab # [ 8.709594] systemd[1]: Reached target Login Prompts.1083vm-test-run-nixbot-gitlab> gitlab # [ 9.028681] cfg80211: Loading compiled-in X.509 certificates for regulatory database1084vm-test-run-nixbot-gitlab> gitlab # [ 9.038059] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'1085vm-test-run-nixbot-gitlab> gitlab # [ 9.039389] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'1086vm-test-run-nixbot-gitlab> gitlab # [ 9.044071] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -21087vm-test-run-nixbot-gitlab> gitlab # [ 9.047250] cfg80211: failed to load regulatory.db1088vm-test-run-nixbot-gitlab> gitlab # [ 8.801604] nginx-pre-start[1267]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok1089vm-test-run-nixbot-gitlab> gitlab # [ 8.805913] nginx-pre-start[1267]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful1090vm-test-run-nixbot-gitlab> gitlab # [ 8.813520] systemd[1]: Started Nginx Web Server.1091vm-test-run-nixbot-gitlab> gitlab # [ 9.077062] 8021q: adding VLAN 0 to HW filter on device eth01092vm-test-run-nixbot-gitlab> gitlab # [ 8.819830] dhcpcd[1249]: eth0: waiting for carrier1093vm-test-run-nixbot-gitlab> gitlab # [ 8.821814] dhcpcd[1249]: eth0: carrier acquired1094vm-test-run-nixbot-gitlab> gitlab # [ 8.831503] dhcpcd[1249]: DUID 00:01:00:01:31:d5:a5:e5:52:54:00:12:34:561095vm-test-run-nixbot-gitlab> gitlab # [ 8.833045] dhcpcd[1249]: eth0: IAID 00:12:34:561096vm-test-run-nixbot-gitlab> gitlab # [ 8.834457] dhcpcd[1249]: eth0: adding address fe80::5054:ff:fe12:34561097vm-test-run-nixbot-gitlab> gitlab # [ 8.839352] redis-server[1270]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1098vm-test-run-nixbot-gitlab> gitlab # [ 8.840877] redis-server[1270]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1099vm-test-run-nixbot-gitlab> gitlab # [ 8.845146] redis-server[1270]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1270, just started1100vm-test-run-nixbot-gitlab> gitlab # [ 8.848273] redis-server[1270]: Configuration loaded1101vm-test-run-nixbot-gitlab> gitlab # [ 8.850741] systemd[1]: Started Redis Server - redis-gitlab.1102vm-test-run-nixbot-gitlab> gitlab # [ 8.851846] redis[1270]: monotonic clock: POSIX clock_gettime1103vm-test-run-nixbot-gitlab> gitlab # [ 8.853488] redis[1270]: Running mode=standalone, port=0.1104vm-test-run-nixbot-gitlab> gitlab # [ 8.855174] redis[1270]: Server initialized1105vm-test-run-nixbot-gitlab> gitlab # [ 8.856435] redis[1270]: Ready to accept connections unix1106vm-test-run-nixbot-gitlab> gitlab # [ 8.857900] redis[1270]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1107vm-test-run-nixbot-gitlab> gitlab # [ 8.867381] postgresql-pre-start[1278]: The files belonging to this database system will be owned by user "postgres".1108vm-test-run-nixbot-gitlab> gitlab # [ 8.869098] postgresql-pre-start[1278]: This user must also own the server process.1109vm-test-run-nixbot-gitlab> gitlab # [ 8.872439] postgresql-pre-start[1278]: The database cluster will be initialized with locale "en_US.UTF-8".1110vm-test-run-nixbot-gitlab> gitlab # [ 8.873798] postgresql-pre-start[1278]: The default database encoding has accordingly been set to "UTF8".1111vm-test-run-nixbot-gitlab> gitlab # [ 8.875888] postgresql-pre-start[1278]: The default text search configuration will be set to "english".1112vm-test-run-nixbot-gitlab> gitlab # [ 8.877208] postgresql-pre-start[1278]: Data page checksums are disabled.1113vm-test-run-nixbot-gitlab> gitlab # [ 8.878184] postgresql-pre-start[1278]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1114vm-test-run-nixbot-gitlab> gitlab # [ 8.878958] postgresql-pre-start[1278]: creating subdirectories ... ok1115vm-test-run-nixbot-gitlab> gitlab # [ 8.880331] postgresql-pre-start[1278]: selecting dynamic shared memory implementation ... posix1116vm-test-run-nixbot-gitlab> gitlab # [ 8.905535] postgresql-pre-start[1278]: selecting default "max_connections" ... 1001117vm-test-run-nixbot-gitlab> gitlab # [ 8.939924] postgresql-pre-start[1278]: selecting default "shared_buffers" ... 128MB1118vm-test-run-nixbot-gitlab> gitlab # [ 9.148705] systemd[1]: Started gitlab-workhorse.service.1119vm-test-run-nixbot-gitlab> gitlab # [ 9.329771] postgresql-pre-start[1278]: selecting default time zone ... UTC1120vm-test-run-nixbot-gitlab> gitlab # [ 9.334899] postgresql-pre-start[1278]: creating configuration files ... ok1121vm-test-run-nixbot-gitlab> gitlab # [ 9.391698] gitlab-workhorse[1305]: time="2026-06-29T21:53:42Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41122vm-test-run-nixbot-gitlab> gitlab # [ 9.392935] gitlab-workhorse[1305]: time="2026-06-29T21:53:42Z" level=info msg="Using redis/go-redis"1123vm-test-run-nixbot-gitlab> gitlab # [ 9.394254] gitlab-workhorse[1305]: time="2026-06-29T21:53:42Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1124vm-test-run-nixbot-gitlab> gitlab # [ 9.430455] dhcpcd[1249]: eth0: soliciting a DHCP lease1125vm-test-run-nixbot-gitlab> gitlab # [ 9.697973] NET: Registered PF_PACKET protocol family1126vm-test-run-nixbot-gitlab> gitlab # [ 9.444961] dhcpcd[1249]: eth0: offered 10.0.2.15 from 10.0.2.21127vm-test-run-nixbot-gitlab> gitlab # [ 9.450289] dhcpcd[1249]: eth0: probing address 10.0.2.15/241128vm-test-run-nixbot-gitlab> gitlab # [ 9.510399] postgresql-pre-start[1278]: running bootstrap script ... ok1129vm-test-run-nixbot-gitlab> gitlab # [ 9.988728] postgresql-pre-start[1278]: performing post-bootstrap initialization ... ok1130vm-test-run-nixbot-gitlab> gitlab # [ 10.453991] postgresql-pre-start[1278]: syncing data to disk ... ok1131vm-test-run-nixbot-gitlab> gitlab # [ 10.456927] postgresql-pre-start[1278]: initdb: warning: enabling "trust" authentication for local connections1132vm-test-run-nixbot-gitlab> gitlab # [ 10.461128] postgresql-pre-start[1278]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1133vm-test-run-nixbot-gitlab> gitlab # [ 10.463971] postgresql-pre-start[1278]: Success. You can now start the database server using:1134vm-test-run-nixbot-gitlab> gitlab # [ 10.468372] postgresql-pre-start[1278]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1135vm-test-run-nixbot-gitlab> gitlab # [ 10.598876] postgres[1331]: [1331] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1136vm-test-run-nixbot-gitlab> gitlab # [ 10.605845] postgres[1331]: [1331] LOG: listening on IPv6 address "::1", port 54321137vm-test-run-nixbot-gitlab> gitlab # [ 10.610624] postgres[1331]: [1331] LOG: listening on IPv4 address "127.0.0.1", port 54321138vm-test-run-nixbot-gitlab> gitlab # [ 10.615740] postgres[1331]: [1331] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1139vm-test-run-nixbot-gitlab> gitlab # [ 10.629932] postgres[1342]: [1342] LOG: database system was shut down at 2026-06-29 21:53:42 GMT1140vm-test-run-nixbot-gitlab> gitlab # [ 10.641267] postgres[1331]: [1331] LOG: database system is ready to accept connections1141vm-test-run-nixbot-gitlab> gitlab # [ 10.651123] systemd[1]: Started PostgreSQL Server.1142vm-test-run-nixbot-gitlab> gitlab # [ 10.658300] systemd[1]: Starting PostgreSQL Setup Scripts...1143vm-test-run-nixbot-gitlab> gitlab # [ 10.840645] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1357]: mkdir -p /var/gitlab/state/home/.ssh: OK1144vm-test-run-nixbot-gitlab> gitlab # [ 10.845698] postgresql-setup-start[1363]: CREATE DATABASE1145vm-test-run-nixbot-gitlab> gitlab # [ 10.851167] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1357]: chmod 700 /var/gitlab/state/home/.ssh: OK1146vm-test-run-nixbot-gitlab> gitlab # [ 10.876544] postgresql-setup-start[1370]: CREATE ROLE1147vm-test-run-nixbot-gitlab> gitlab # [ 10.895694] postgresql-setup-start[1378]: ALTER DATABASE1148vm-test-run-nixbot-gitlab> gitlab # [ 10.906213] systemd[1]: Finished gitlab-config.service.1149vm-test-run-nixbot-gitlab> gitlab # [ 10.909721] systemd[1]: Started gitaly.service.1150vm-test-run-nixbot-gitlab> gitlab # [ 10.924554] postgresql-setup-start[1388]: CREATE ROLE1151vm-test-run-nixbot-gitlab> gitlab # [ 10.929097] systemd[1]: Finished PostgreSQL Setup Scripts.1152vm-test-run-nixbot-gitlab> gitlab # [ 10.933668] systemd[1]: Reached target PostgreSQL.1153vm-test-run-nixbot-gitlab> gitlab # [ 10.936616] systemd[1]: Starting gitlab-db-config.service...1154vm-test-run-nixbot-gitlab> gitlab # [ 10.941845] systemd[1]: Starting gitlab-postgresql.service...1155vm-test-run-nixbot-gitlab> gitlab # [ 11.048774] gitlab-postgresql-start[1399]: CREATE DATABASE1156vm-test-run-nixbot-gitlab> gitlab # [ 11.101323] gitlab-postgresql-start[1404]: CREATE EXTENSION1157vm-test-run-nixbot-gitlab> gitlab # [ 11.154665] gitlab-postgresql-start[1406]: CREATE EXTENSION1158vm-test-run-nixbot-gitlab> gitlab # [ 11.159268] systemd[1]: Finished gitlab-postgresql.service.1159vm-test-run-nixbot-gitlab> gitlab # [ 11.162418] dhcpcd[1249]: eth0: soliciting an IPv6 router1160vm-test-run-nixbot-gitlab> gitlab # [ 11.164483] dhcpcd[1249]: eth0: Router Advertisement from fe80::21161vm-test-run-nixbot-gitlab> gitlab # [ 11.166308] dhcpcd[1249]: eth0: adding address fec0::5054:ff:fe12:3456/641162vm-test-run-nixbot-gitlab> gitlab # [ 11.168056] dhcpcd[1249]: eth0: adding route to fec0::/641163vm-test-run-nixbot-gitlab> gitlab # [ 11.169541] dhcpcd[1249]: eth0: adding default route via fe80::21164vm-test-run-nixbot-gitlab> gitlab # [ 11.468831] gitaly[1387]: time="2026-06-29T21:53:44.460Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13871165vm-test-run-nixbot-gitlab> gitlab # [ 11.471747] gitaly[1387]: time="2026-06-29T21:53:44.460Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13871166vm-test-run-nixbot-gitlab> gitlab # [ 11.474109] gitaly[1387]: time="2026-06-29T21:53:44.460Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13871167vm-test-run-nixbot-gitlab> gitlab # [ 11.475503] gitaly[1387]: time="2026-06-29T21:53:44.461Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13871168vm-test-run-nixbot-gitlab> gitlab # [ 11.477887] gitaly[1387]: time="2026-06-29T21:53:44.461Z" level=info msg="Starting Gitaly" pid=1387 version=18.11.41169vm-test-run-nixbot-gitlab> gitlab # [ 11.694645] gitaly[1387]: time="2026-06-29T21:53:44.687Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=225 pid=13871170vm-test-run-nixbot-gitlab> gitlab # [ 11.698219] gitaly[1387]: time="2026-06-29T21:53:44.690Z" level=info msg="finished initializing bootstrap" duration_ms=2 pid=13871171vm-test-run-nixbot-gitlab> gitlab # [ 11.701225] gitaly[1387]: time="2026-06-29T21:53:44.690Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13871172vm-test-run-nixbot-gitlab> gitlab # [ 11.704264] gitaly[1387]: time="2026-06-29T21:53:44.690Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1387/git-exec-1895731829.d/git pid=13871173vm-test-run-nixbot-gitlab> gitlab # [ 11.708260] gitaly[1387]: time="2026-06-29T21:53:44.698Z" level=info msg="finished detecting git version" duration_ms=7 pid=13871174vm-test-run-nixbot-gitlab> gitlab # [ 11.711501] gitaly[1387]: time="2026-06-29T21:53:44.698Z" level=info msg="using Git version" pid=1387 version=2.53.g1175vm-test-run-nixbot-gitlab> gitlab # [ 11.714321] gitaly[1387]: time="2026-06-29T21:53:44.699Z" level=info msg="clearing disk cache object folder" pid=1387 storage=default1176vm-test-run-nixbot-gitlab> gitlab # [ 11.717605] gitaly[1387]: time="2026-06-29T21:53:44.699Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1101347460 pid=1387 storage=default1177vm-test-run-nixbot-gitlab> gitlab # [ 11.722541] gitaly[1387]: time="2026-06-29T21:53:44.699Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1101347460 pid=1387 storage=default1178vm-test-run-nixbot-gitlab> gitlab # [ 11.728208] gitaly[1387]: time="2026-06-29T21:53:44.699Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13871179vm-test-run-nixbot-gitlab> gitlab # [ 11.729508] gitaly[1387]: time="2026-06-29T21:53:44.699Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1101347460 clear_duration_ms=0 pid=1387 storage=default1180vm-test-run-nixbot-gitlab> gitlab # [ 11.730841] gitaly[1387]: time="2026-06-29T21:53:44.699Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13871181vm-test-run-nixbot-gitlab> gitlab # [ 11.779217] gitaly[1387]: time="2026-06-29T21:53:44.771Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13871182vm-test-run-nixbot-gitlab> gitlab # [ 11.781704] gitaly[1387]: time="2026-06-29T21:53:44.772Z" level=info msg="counting repositories" pid=13871183vm-test-run-nixbot-gitlab> gitlab # [ 11.783712] gitaly[1387]: time="2026-06-29T21:53:44.772Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1387 schema=unix1184vm-test-run-nixbot-gitlab> gitlab # [ 11.785648] gitaly[1387]: time="2026-06-29T21:53:44.772Z" level=info msg="finished tempdir cleaner walk" pid=1387 storage=default time_ms=01185vm-test-run-nixbot-gitlab> gitlab # [ 11.787394] gitaly[1387]: time="2026-06-29T21:53:44.773Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1387/sock.d/intern pid=1387 schema=unix1186vm-test-run-nixbot-gitlab> gitlab # [ 11.789431] gitaly[1387]: time="2026-06-29T21:53:44.773Z" level=info msg="starting to count repositories" pid=1387 storage_path=/var/gitlab/state/repositories1187vm-test-run-nixbot-gitlab> gitlab # [ 11.791258] gitaly[1387]: time="2026-06-29T21:53:44.774Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1387 storage_path=/var/gitlab/state/repositories1188vm-test-run-nixbot-gitlab> gitlab # [ 11.792373] gitaly[1387]: time="2026-06-29T21:53:44.774Z" level=info msg="completed counting all repositories" counting_duration=1 pid=13871189vm-test-run-nixbot-gitlab> gitlab # [ 11.806790] gitaly[1387]: time="2026-06-29T21:53:44.799Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13871190vm-test-run-nixbot-gitlab> gitlab # [ 11.808145] gitaly[1387]: time="2026-06-29T21:53:44.799Z" level=info msg="Started Gitaly" duration_ms=338 pid=13871191vm-test-run-nixbot-gitlab> gitlab # [ 11.808941] gitaly[1387]: time="2026-06-29T21:53:44.800Z" level=info msg="maintenance: daily scheduled" pid=1387 scheduled="2026-06-30 12:00:00 +0000 UTC"1192vm-test-run-nixbot-gitlab> gitlab # [ 12.356120] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1394]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1193vm-test-run-nixbot-gitlab> gitlab # [ 12.359701] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1394]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1194vm-test-run-nixbot-gitlab> gitlab # [ 12.381160] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1394]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1195vm-test-run-nixbot-gitlab> gitlab # [ 13.686690] gitaly[1387]: time="2026-06-29T21:53:46.678Z" level=info msg="License database preloaded" duration_ms=2217 pid=13871196vm-test-run-nixbot-gitlab> gitlab # [ 15.417972] dhcpcd[1249]: eth0: leased 10.0.2.15 for 86400 seconds1197vm-test-run-nixbot-gitlab> gitlab # [ 15.420956] dhcpcd[1249]: eth0: adding route to 10.0.2.0/241198vm-test-run-nixbot-gitlab> gitlab # [ 15.422144] dhcpcd[1249]: eth0: adding default route via 10.0.2.21199vm-test-run-nixbot-gitlab> gitlab # [ 15.536962] systemd[1]: Started DHCP Client.1200vm-test-run-nixbot-gitlab> gitlab # [ 15.540398] systemd[1]: Reached target Network is Online.1201vm-test-run-nixbot-gitlab> gitlab # [ 45.376424] postgres[1783]: [1783] ERROR: relation "feature_gates" does not exist at character 5231202vm-test-run-nixbot-gitlab> gitlab # [ 45.382566] postgres[1783]: [1783] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1203vm-test-run-nixbot-gitlab> gitlab # [ 45.387922] postgres[1783]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1204vm-test-run-nixbot-gitlab> gitlab # [ 45.393721] postgres[1783]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1205vm-test-run-nixbot-gitlab> gitlab # [ 45.399177] postgres[1783]: attidentity AS identity,1206vm-test-run-nixbot-gitlab> gitlab # [ 45.403260] postgres[1783]: attgenerated as attgenerated1207vm-test-run-nixbot-gitlab> gitlab # [ 45.407477] postgres[1783]: FROM pg_attribute a1208vm-test-run-nixbot-gitlab> gitlab # [ 45.410822] postgres[1783]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1209vm-test-run-nixbot-gitlab> gitlab # [ 45.416671] postgres[1783]: LEFT JOIN pg_type t ON a.atttypid = t.oid1210vm-test-run-nixbot-gitlab> gitlab # [ 45.421181] postgres[1783]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1211vm-test-run-nixbot-gitlab> gitlab # [ 45.427767] postgres[1783]: WHERE a.attrelid = '"feature_gates"'::regclass1212vm-test-run-nixbot-gitlab> gitlab # [ 45.431751] postgres[1783]: AND a.attnum > 0 AND NOT a.attisdropped1213vm-test-run-nixbot-gitlab> gitlab # [ 45.435218] postgres[1783]: ORDER BY a.attnum1214vm-test-run-nixbot-gitlab> gitlab # [ 45.437658] postgres[1783]:1215vm-test-run-nixbot-gitlab> gitlab # [ 45.589306] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1394]: W, [2026-06-29T21:54:18.581292 #1394] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1216vm-test-run-nixbot-gitlab> gitlab # [ 49.474568] gitlab-workhorse[1305]: time="2026-06-29T21:54:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1217vm-test-run-nixbot-gitlab> gitlab # [ 51.293630] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1847]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1218vm-test-run-nixbot-gitlab> gitlab # [ 51.302381] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1847]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1219vm-test-run-nixbot-gitlab> gitlab # [ 59.485804] gitlab-workhorse[1305]: time="2026-06-29T21:54:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1220vm-test-run-nixbot-gitlab> gitlab # [ 69.138522] postgres[1922]: [1922] ERROR: relation "table_without_models" does not exist1221vm-test-run-nixbot-gitlab> gitlab # [ 69.143690] postgres[1922]: [1922] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1222vm-test-run-nixbot-gitlab> gitlab # [ 69.495951] gitlab-workhorse[1305]: time="2026-06-29T21:54:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1223vm-test-run-nixbot-gitlab> gitlab # [ 79.497445] gitlab-workhorse[1305]: time="2026-06-29T21:54:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1224vm-test-run-nixbot-gitlab> gitlab # [ 85.361743] systemd[1]: Finished gitlab-db-config.service.1225vm-test-run-nixbot-gitlab> gitlab # [ 85.369884] systemd[1]: Started gitlab-sidekiq.service.1226vm-test-run-nixbot-gitlab> gitlab # [ 85.376279] systemd[1]: Starting gitlab.service...1227vm-test-run-nixbot-gitlab> gitlab # [ 85.725615] bundle[2143]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1228vm-test-run-nixbot-gitlab> gitlab # [ 85.727733] bundle[2143]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1229vm-test-run-nixbot-gitlab> gitlab # [ 85.729738] bundle[2143]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1230vm-test-run-nixbot-gitlab> gitlab # [ 88.797728] sidekiq-cluster[2142]: {"severity":"INFO","time":"2026-06-29T21:55:01.771Z","message":"Starting cluster with 1 processes"}1231vm-test-run-nixbot-gitlab> gitlab # [ 88.921496] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.890Z","pid":2143,"message":"* Enabling systemd notification integration"}1232vm-test-run-nixbot-gitlab> gitlab # [ 88.925342] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.913Z","pid":2143,"message":"Puma starting in cluster mode..."}1233vm-test-run-nixbot-gitlab> gitlab # [ 88.928379] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.913Z","pid":2143,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1234vm-test-run-nixbot-gitlab> gitlab # [ 88.931701] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.913Z","pid":2143,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1235vm-test-run-nixbot-gitlab> gitlab # [ 88.934539] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.913Z","pid":2143,"message":"* Min threads: 0"}1236vm-test-run-nixbot-gitlab> gitlab # [ 88.936854] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.913Z","pid":2143,"message":"* Max threads: 4"}1237vm-test-run-nixbot-gitlab> gitlab # [ 88.941657] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.913Z","pid":2143,"message":"* Environment: production"}1238vm-test-run-nixbot-gitlab> gitlab # [ 88.945398] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.914Z","pid":2143,"message":"* Master PID: 2143"}1239vm-test-run-nixbot-gitlab> gitlab # [ 88.948412] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.914Z","pid":2143,"message":"* Workers: 2"}1240vm-test-run-nixbot-gitlab> gitlab # [ 88.951456] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.914Z","pid":2143,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1241vm-test-run-nixbot-gitlab> gitlab # [ 88.954562] bundle[2143]: {"timestamp":"2026-06-29T21:55:01.914Z","pid":2143,"message":"* Preloading application"}1242vm-test-run-nixbot-gitlab> gitlab # [ 89.506886] gitlab-workhorse[1305]: time="2026-06-29T21:55:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1243vm-test-run-nixbot-gitlab> gitlab # [ 99.517777] gitlab-workhorse[1305]: time="2026-06-29T21:55:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1244vm-test-run-nixbot-gitlab> gitlab # [ 109.528228] gitlab-workhorse[1305]: time="2026-06-29T21:55:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1245vm-test-run-nixbot-gitlab> gitlab # [ 114.350213] bundle[2143]: W, [2026-06-29T21:55:27.342275 #2143] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1246vm-test-run-nixbot-gitlab> gitlab # [ 117.048185] sidekiq-cluster[2173]: 2026-06-29T21:55:30.039Z pid=2173 tid=u0l INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2173, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430563328}1247vm-test-run-nixbot-gitlab> gitlab # [ 117.241328] sidekiq-cluster[2173]: W, [2026-06-29T21:55:30.234192 #2173] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1248vm-test-run-nixbot-gitlab> gitlab # [ 117.328968] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:30.321Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1249vm-test-run-nixbot-gitlab> gitlab # [ 119.389875] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.381Z","message":"GitLab reliable fetch activated!"}1250vm-test-run-nixbot-gitlab> gitlab # [ 119.396877] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.382Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1251vm-test-run-nixbot-gitlab> gitlab # [ 119.416407] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.409Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1252vm-test-run-nixbot-gitlab> gitlab # [ 119.426421] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.413Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1253vm-test-run-nixbot-gitlab> gitlab # [ 119.428982] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.417Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [ 119.431926] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.420Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [ 119.435288] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.423Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [ 119.438301] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.426Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [ 119.440926] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.428Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [ 119.443479] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.430Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [ 119.445887] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.433Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [ 119.447472] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.435Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [ 119.449661] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.438Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [ 119.451887] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.441Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [ 119.454516] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.443Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [ 119.456694] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.445Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [ 119.458339] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.448Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [ 119.460058] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.450Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [ 119.460913] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.453Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [ 119.463306] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.456Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [ 119.466278] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.459Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [ 119.469206] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.462Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [ 119.472093] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.465Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [ 119.475099] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.468Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [ 119.477802] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.470Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [ 119.480501] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.473Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [ 119.482937] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.475Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [ 119.485503] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.478Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [ 119.488668] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.481Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [ 119.491481] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.484Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [ 119.494269] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.487Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [ 119.496954] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.490Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [ 119.499704] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.492Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [ 119.502188] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.495Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [ 119.504985] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.498Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [ 119.507759] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.500Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1285vm-test-run-nixbot-gitlab> gitlab # [ 119.510477] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.503Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1286vm-test-run-nixbot-gitlab> gitlab # [ 119.512947] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.506Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1287vm-test-run-nixbot-gitlab> gitlab # [ 119.515329] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.508Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1288vm-test-run-nixbot-gitlab> gitlab # [ 119.517749] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.510Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1289vm-test-run-nixbot-gitlab> gitlab # [ 119.520390] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.513Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1290vm-test-run-nixbot-gitlab> gitlab # [ 119.522729] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.515Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1291vm-test-run-nixbot-gitlab> gitlab # [ 119.525166] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.518Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1292vm-test-run-nixbot-gitlab> gitlab # [ 119.527747] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.520Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1293vm-test-run-nixbot-gitlab> gitlab # [ 119.530073] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.523Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1294vm-test-run-nixbot-gitlab> gitlab # [ 119.532712] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.525Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1295vm-test-run-nixbot-gitlab> gitlab # [ 119.535345] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.528Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1296vm-test-run-nixbot-gitlab> gitlab # [ 119.538213] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.531Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1297vm-test-run-nixbot-gitlab> gitlab # [ 119.540592] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.533Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1298vm-test-run-nixbot-gitlab> gitlab # [ 119.542886] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.535Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1299vm-test-run-nixbot-gitlab> gitlab # [ 119.545523] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.538Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1300vm-test-run-nixbot-gitlab> gitlab # [ 119.548236] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.541Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1301vm-test-run-nixbot-gitlab> gitlab # [ 119.550949] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.544Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1302vm-test-run-nixbot-gitlab> gitlab # [ 119.553825] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.546Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1303vm-test-run-nixbot-gitlab> gitlab # [ 119.556654] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.549Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1304vm-test-run-nixbot-gitlab> gitlab # [ 119.567390] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.560Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1305vm-test-run-nixbot-gitlab> gitlab # [ 119.570521] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.563Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1306vm-test-run-nixbot-gitlab> gitlab # [ 119.573271] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.566Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1307vm-test-run-nixbot-gitlab> gitlab # [ 119.575909] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.568Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1308vm-test-run-nixbot-gitlab> gitlab # [ 119.578648] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.571Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1309vm-test-run-nixbot-gitlab> gitlab # [ 119.581645] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.574Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1310vm-test-run-nixbot-gitlab> gitlab # [ 119.584049] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.577Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1311vm-test-run-nixbot-gitlab> gitlab # [ 119.586818] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.579Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1312vm-test-run-nixbot-gitlab> gitlab # [ 119.589758] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.582Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1313vm-test-run-nixbot-gitlab> gitlab # [ 119.592093] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.585Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1314vm-test-run-nixbot-gitlab> gitlab # [ 119.595068] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.588Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1315vm-test-run-nixbot-gitlab> gitlab # [ 119.598077] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.591Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1316vm-test-run-nixbot-gitlab> gitlab # [ 119.600863] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.593Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1317vm-test-run-nixbot-gitlab> gitlab # [ 119.603833] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.596Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1318vm-test-run-nixbot-gitlab> gitlab # [ 119.606212] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.599Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1319vm-test-run-nixbot-gitlab> gitlab # [ 119.608565] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.601Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1320vm-test-run-nixbot-gitlab> gitlab # [ 119.611237] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.603Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1321vm-test-run-nixbot-gitlab> gitlab # [ 119.614759] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.606Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1322vm-test-run-nixbot-gitlab> gitlab # [ 119.618831] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.608Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1323vm-test-run-nixbot-gitlab> gitlab # [ 119.622384] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.611Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1324vm-test-run-nixbot-gitlab> gitlab # [ 119.626237] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.613Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1325vm-test-run-nixbot-gitlab> gitlab # [ 119.630041] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.615Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1326vm-test-run-nixbot-gitlab> gitlab # [ 119.701442] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.618Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1327vm-test-run-nixbot-gitlab> gitlab # [ 119.703492] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.620Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1328vm-test-run-nixbot-gitlab> gitlab # [ 119.705253] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.623Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1329vm-test-run-nixbot-gitlab> gitlab # [ 119.706994] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.625Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1330vm-test-run-nixbot-gitlab> gitlab # [ 119.708566] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.627Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1331vm-test-run-nixbot-gitlab> gitlab # [ 119.710364] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.630Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1332vm-test-run-nixbot-gitlab> gitlab # [ 119.712142] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.632Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1333vm-test-run-nixbot-gitlab> gitlab # [ 119.713576] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.635Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1334vm-test-run-nixbot-gitlab> gitlab # [ 119.715236] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.638Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1335vm-test-run-nixbot-gitlab> gitlab # [ 119.717106] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.641Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1336vm-test-run-nixbot-gitlab> gitlab # [ 119.718632] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.643Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1337vm-test-run-nixbot-gitlab> gitlab # [ 119.720234] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.646Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1338vm-test-run-nixbot-gitlab> gitlab # [ 119.721901] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.666Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1339vm-test-run-nixbot-gitlab> gitlab # [ 119.723305] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.669Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1340vm-test-run-nixbot-gitlab> gitlab # [ 119.724959] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.672Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1341vm-test-run-nixbot-gitlab> gitlab # [ 119.726265] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.674Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1342vm-test-run-nixbot-gitlab> gitlab # [ 119.727438] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.677Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1343vm-test-run-nixbot-gitlab> gitlab # [ 119.728864] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.680Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1344vm-test-run-nixbot-gitlab> gitlab # [ 119.730493] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.682Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1345vm-test-run-nixbot-gitlab> gitlab # [ 119.731794] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.685Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1346vm-test-run-nixbot-gitlab> gitlab # [ 119.732704] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.689Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1347vm-test-run-nixbot-gitlab> gitlab # [ 119.733646] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.692Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1348vm-test-run-nixbot-gitlab> gitlab # [ 119.734794] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.695Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1349vm-test-run-nixbot-gitlab> gitlab # [ 119.735721] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.697Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1350vm-test-run-nixbot-gitlab> gitlab # [ 119.736886] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.708Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1351vm-test-run-nixbot-gitlab> gitlab # [ 119.738165] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.710Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1352vm-test-run-nixbot-gitlab> gitlab # [ 119.738967] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.713Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1353vm-test-run-nixbot-gitlab> gitlab # [ 119.739870] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.716Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1354vm-test-run-nixbot-gitlab> gitlab # [ 119.740766] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.720Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1355vm-test-run-nixbot-gitlab> gitlab # [ 119.742825] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.723Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1356vm-test-run-nixbot-gitlab> gitlab # [ 119.746874] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.726Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1357vm-test-run-nixbot-gitlab> gitlab # [ 119.750982] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.728Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1358vm-test-run-nixbot-gitlab> gitlab # [ 119.754168] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.731Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1359vm-test-run-nixbot-gitlab> gitlab # [ 119.758288] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.734Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1360vm-test-run-nixbot-gitlab> gitlab # [ 119.761949] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:32.736Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1361vm-test-run-nixbot-gitlab> gitlab # [ 119.837354] gitlab-workhorse[1305]: time="2026-06-29T21:55:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1362vm-test-run-nixbot-gitlab> gitlab # [ 122.173469] sidekiq-cluster[2173]: InitializerConnections Query: SET client_min_messages TO 'warning'1363vm-test-run-nixbot-gitlab> gitlab # [ 122.176791] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1364vm-test-run-nixbot-gitlab> gitlab # [ 122.181474] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1365vm-test-run-nixbot-gitlab> gitlab # [ 122.184184] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1366vm-test-run-nixbot-gitlab> gitlab # [ 122.186913] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1367vm-test-run-nixbot-gitlab> gitlab # [ 122.189731] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1368vm-test-run-nixbot-gitlab> gitlab # [ 122.191447] sidekiq-cluster[2173]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1369vm-test-run-nixbot-gitlab> gitlab # [ 122.214638] sidekiq-cluster[2173]: InitializerConnections Query: SET standard_conforming_strings = on1370vm-test-run-nixbot-gitlab> gitlab # [ 122.216476] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1371vm-test-run-nixbot-gitlab> gitlab # [ 122.218772] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1372vm-test-run-nixbot-gitlab> gitlab # [ 122.220373] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1373vm-test-run-nixbot-gitlab> gitlab # [ 122.222579] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1374vm-test-run-nixbot-gitlab> gitlab # [ 122.224681] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1375vm-test-run-nixbot-gitlab> gitlab # [ 122.226221] sidekiq-cluster[2173]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1376vm-test-run-nixbot-gitlab> gitlab # [ 122.249270] sidekiq-cluster[2173]: InitializerConnections Query: SET intervalstyle = iso_86011377vm-test-run-nixbot-gitlab> gitlab # [ 122.250429] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1378vm-test-run-nixbot-gitlab> gitlab # [ 122.252146] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1379vm-test-run-nixbot-gitlab> gitlab # [ 122.254281] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1380vm-test-run-nixbot-gitlab> gitlab # [ 122.255950] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1381vm-test-run-nixbot-gitlab> gitlab # [ 122.257336] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1382vm-test-run-nixbot-gitlab> gitlab # [ 122.258988] sidekiq-cluster[2173]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1383vm-test-run-nixbot-gitlab> gitlab # [ 122.269576] sidekiq-cluster[2173]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname1384vm-test-run-nixbot-gitlab> gitlab # [ 122.272797] sidekiq-cluster[2173]: FROM pg_type as t1385vm-test-run-nixbot-gitlab> gitlab # [ 122.274084] sidekiq-cluster[2173]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')1386vm-test-run-nixbot-gitlab> gitlab # [ 122.277443] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1387vm-test-run-nixbot-gitlab> gitlab # [ 122.278710] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1388vm-test-run-nixbot-gitlab> gitlab # [ 122.279862] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1389vm-test-run-nixbot-gitlab> gitlab # [ 122.280738] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1390vm-test-run-nixbot-gitlab> gitlab # [ 122.281902] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1391vm-test-run-nixbot-gitlab> gitlab # [ 122.283076] sidekiq-cluster[2173]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1392vm-test-run-nixbot-gitlab> gitlab # [ 122.284056] sidekiq-cluster[2173]: InitializerConnections Query: SET SESSION timezone TO 'UTC'1393vm-test-run-nixbot-gitlab> gitlab # [ 122.284814] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1394vm-test-run-nixbot-gitlab> gitlab # [ 122.285693] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1395vm-test-run-nixbot-gitlab> gitlab # [ 122.286824] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1396vm-test-run-nixbot-gitlab> gitlab # [ 122.287961] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1397vm-test-run-nixbot-gitlab> gitlab # [ 122.289150] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1398vm-test-run-nixbot-gitlab> gitlab # [ 122.289985] sidekiq-cluster[2173]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1399vm-test-run-nixbot-gitlab> gitlab # [ 122.308273] sidekiq-cluster[2173]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics'1400vm-test-run-nixbot-gitlab> gitlab # [ 122.309347] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1401vm-test-run-nixbot-gitlab> gitlab # [ 122.310172] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1402vm-test-run-nixbot-gitlab> gitlab # [ 122.311078] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1403vm-test-run-nixbot-gitlab> gitlab # [ 122.312716] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1404vm-test-run-nixbot-gitlab> gitlab # [ 122.313897] sidekiq-cluster[2173]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1405vm-test-run-nixbot-gitlab> gitlab # [ 122.315938] sidekiq-cluster[2173]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1406vm-test-run-nixbot-gitlab> gitlab # [ 129.837888] gitlab-workhorse[1305]: time="2026-06-29T21:55:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1407vm-test-run-nixbot-gitlab> gitlab # [ 139.841553] gitlab-workhorse[1305]: time="2026-06-29T21:55:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1408vm-test-run-nixbot-gitlab> gitlab # [ 142.372650] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:55.364Z","message":"Booted Rails 7.2.3 application in production environment"}1409vm-test-run-nixbot-gitlab> gitlab # [ 142.375207] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:55.365Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1410vm-test-run-nixbot-gitlab> gitlab # [ 142.376921] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:55.366Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1411vm-test-run-nixbot-gitlab> gitlab # [ 142.378595] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:55.366Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1412vm-test-run-nixbot-gitlab> gitlab # [ 142.380410] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:55.369Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1413vm-test-run-nixbot-gitlab> gitlab # [ 143.564985] bundle[2143]: {"timestamp":"2026-06-29T21:55:56.556Z","pid":2143,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1414vm-test-run-nixbot-gitlab> gitlab # [ 143.569118] bundle[2143]: {"timestamp":"2026-06-29T21:55:56.557Z","pid":2143,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1415vm-test-run-nixbot-gitlab> gitlab # [ 143.571981] bundle[2143]: {"timestamp":"2026-06-29T21:55:56.557Z","pid":2143,"message":"! #\u003cThread:0x0000797f5799a520 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1416vm-test-run-nixbot-gitlab> gitlab # [ 143.574917] bundle[2143]: {"timestamp":"2026-06-29T21:55:56.557Z","pid":2143,"message":"! #\u003cThread:0x0000797f57bdae98 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1417vm-test-run-nixbot-gitlab> gitlab # [ 143.578230] bundle[2143]: {"timestamp":"2026-06-29T21:55:56.557Z","pid":2143,"message":"Use Ctrl-C to stop"}1418vm-test-run-nixbot-gitlab> gitlab # [ 143.620725] bundle[2143]: {"timestamp":"2026-06-29T21:55:56.558Z","pid":2143,"message":"Sending status to systemd every 1.0 sec"}1419vm-test-run-nixbot-gitlab> gitlab # [ 143.863657] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:56.856Z","message":"Cleaning working queues"}1420vm-test-run-nixbot-gitlab> gitlab # [ 143.865440] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:56.857Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4d0ba380ac194e47bbb3636c","created_at":"2026-06-29T21:54:57.662Z","trace_propagation_headers":{"sentry-trace":"41321bef216040c6ab83a13a63c1f55a-84200ce4161147af","baggage":"sentry-trace_id=41321bef216040c6ab83a13a63c1f55a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"71362c8b2c9bab9a268bf4397f97c25f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T21:54:57.673Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4d0ba380ac194e47bbb3636c: start","job_status":"start","queue_duration_s":59.184279,"scheduling_latency_s":59.184333}1421vm-test-run-nixbot-gitlab> gitlab # [ 143.979361] bundle[2143]: {"timestamp":"2026-06-29T21:55:56.970Z","pid":2143,"message":"- Worker 1 (PID: 2697) booted in 0.28s, phase: 0"}1422vm-test-run-nixbot-gitlab> gitlab # [ 144.015964] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:55:57.008Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4d0ba380ac194e47bbb3636c","created_at":"2026-06-29T21:54:57.662Z","trace_propagation_headers":{"sentry-trace":"41321bef216040c6ab83a13a63c1f55a-84200ce4161147af","baggage":"sentry-trace_id=41321bef216040c6ab83a13a63c1f55a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"71362c8b2c9bab9a268bf4397f97c25f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T21:54:57.673Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4d0ba380ac194e47bbb3636c: done: 0.151084 sec","job_status":"done","queue_duration_s":59.184279,"scheduling_latency_s":59.184333,"redis_calls":24,"redis_duration_s":0.006292,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003141,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001969,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001182,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.077237192,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.151084,"completed_at":"2026-06-29T21:55:57.008Z","load_balancing_strategy":"primary","db_duration_s":0.044457,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1423vm-test-run-nixbot-gitlab> gitlab # [ 144.068497] bundle[2143]: {"timestamp":"2026-06-29T21:55:57.061Z","pid":2143,"message":"- Worker 0 (PID: 2693) booted in 0.4s, phase: 0"}1424vm-test-run-nixbot-gitlab> gitlab # [ 144.072977] systemd[1]: Started gitlab.service.1425vm-test-run-nixbot-gitlab> gitlab # [ 144.074658] systemd[1]: Reached target Common target for all GitLab services..1426vm-test-run-nixbot-gitlab> gitlab # [ 144.088870] systemd[1]: Starting setup-gitlab.service...1427vm-test-run-nixbot-gitlab> gitlab # [ 144.112121] setup-gitlab-start[2738]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1428vm-test-run-nixbot-gitlab> gitlab # [ 146.628053] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:55:59 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 24741429vm-test-run-nixbot-gitlab> gitlab # [ 146.641857] setup-gitlab-start[2757]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1430vm-test-run-nixbot-gitlab> gitlab # [ 146.643449] setup-gitlab-start[2758]: ++ jq -r .access_token1431vm-test-run-nixbot-gitlab> gitlab # [ 148.075821] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:56:01 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 14111432vm-test-run-nixbot-gitlab> gitlab # [ 148.087174] setup-gitlab-start[2738]: + BEARER=41e5b04f9a22f1e7a2297c658842ad5c3d5a9a223cbce6be5ef13fa861567af61433vm-test-run-nixbot-gitlab> gitlab # [ 148.092842] setup-gitlab-start[2738]: + AUTH='Authorization: Bearer 41e5b04f9a22f1e7a2297c658842ad5c3d5a9a223cbce6be5ef13fa861567af6'1434vm-test-run-nixbot-gitlab> gitlab # [ 148.099533] setup-gitlab-start[2738]: + curl -fs -X PUT -H 'Authorization: Bearer 41e5b04f9a22f1e7a2297c658842ad5c3d5a9a223cbce6be5ef13fa861567af6' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1435vm-test-run-nixbot-gitlab> gitlab # [ 148.879760] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:56:01 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 7671436vm-test-run-nixbot-gitlab> gitlab # [ 148.883231] setup-gitlab-start[2738]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1437vm-test-run-nixbot-gitlab> gitlab # [ 148.902349] runuser[2780]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1438vm-test-run-nixbot-gitlab> gitlab # [ 149.285949] setup-gitlab-start[2781]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1439vm-test-run-nixbot-gitlab> gitlab # [ 149.289475] setup-gitlab-start[2781]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1440vm-test-run-nixbot-gitlab> gitlab # [ 149.292297] setup-gitlab-start[2781]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1441vm-test-run-nixbot-gitlab> gitlab # [ 149.871368] setup-gitlab-start[2781]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1442vm-test-run-nixbot-gitlab> gitlab # [ 149.875522] setup-gitlab-start[2781]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1443vm-test-run-nixbot-gitlab> gitlab # [ 149.878262] setup-gitlab-start[2781]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1444vm-test-run-nixbot-gitlab> gitlab # [ 156.050591] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:09.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"7897dfbf033065928918bed3","created_at":"2026-06-29T21:56:08.999Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4d131c022ce1cfccc5feaa740d9e12a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.038Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-7897dfbf033065928918bed3: start","job_status":"start","queue_duration_s":0.003237,"scheduling_latency_s":0.003289}1445vm-test-run-nixbot-gitlab> gitlab # [ 157.016802] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.009Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"7897dfbf033065928918bed3","created_at":"2026-06-29T21:56:08.999Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4d131c022ce1cfccc5feaa740d9e12a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.038Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-7897dfbf033065928918bed3: done: 0.96733 sec","job_status":"done","queue_duration_s":0.003237,"scheduling_latency_s":0.003289,"redis_calls":14,"redis_duration_s":0.022068,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.007975,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00948,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004613,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.06,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.405676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002113676,"exclusive_lock_hold_duration_s":0.837919903,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.96733,"completed_at":"2026-06-29T21:56:10.009Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.124944,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446vm-test-run-nixbot-gitlab> gitlab # [ 157.023526] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.015Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"622ef3b018338e447b3bbc4e","created_at":"2026-06-29T21:56:09.105Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa5948d9350938bb13884a8ca2daccbc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T21:56:09.131Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-622ef3b018338e447b3bbc4e: start","job_status":"start","queue_duration_s":0.88336,"scheduling_latency_s":0.883385}1447vm-test-run-nixbot-gitlab> gitlab # [ 157.161294] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.153Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1448vm-test-run-nixbot-gitlab> gitlab # [ 157.178258] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.170Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"622ef3b018338e447b3bbc4e","created_at":"2026-06-29T21:56:09.105Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa5948d9350938bb13884a8ca2daccbc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T21:56:09.131Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-622ef3b018338e447b3bbc4e: done: 0.155494 sec","job_status":"done","queue_duration_s":0.88336,"scheduling_latency_s":0.883385,"redis_calls":10,"redis_duration_s":0.120798,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.117595,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00171,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000945651,"duration_s":0.155494,"completed_at":"2026-06-29T21:56:10.170Z","load_balancing_strategy":"primary","db_duration_s":0.002283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449vm-test-run-nixbot-gitlab> gitlab # [ 157.246737] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.239Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"cdd4ecf952cced9bc7146638","created_at":"2026-06-29T21:56:09.151Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7df376f7aece45ad0728db743b4fbbf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.171Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-cdd4ecf952cced9bc7146638: start","job_status":"start","queue_duration_s":1.067726,"scheduling_latency_s":1.067792}1450vm-test-run-nixbot-gitlab> gitlab # [ 157.349494] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.341Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"cdd4ecf952cced9bc7146638","created_at":"2026-06-29T21:56:09.151Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7df376f7aece45ad0728db743b4fbbf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.171Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-cdd4ecf952cced9bc7146638: done: 0.102821 sec","job_status":"done","queue_duration_s":1.067726,"scheduling_latency_s":1.067792,"redis_calls":16,"redis_duration_s":0.008998,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.004026,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003373,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001599,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001181435,"duration_s":0.102821,"completed_at":"2026-06-29T21:56:10.341Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0146,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451vm-test-run-nixbot-gitlab> gitlab # [ 157.440341] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.431Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"70623d97a6d4808aebceccb7","created_at":"2026-06-29T21:56:09.191Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e821ed35a4cb86950ce7ac1ea4822fe2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.203Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-70623d97a6d4808aebceccb7: start","job_status":"start","queue_duration_s":1.228452,"scheduling_latency_s":1.228521}1452vm-test-run-nixbot-gitlab> gitlab # [ 157.510314] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.502Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"70623d97a6d4808aebceccb7","created_at":"2026-06-29T21:56:09.191Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e821ed35a4cb86950ce7ac1ea4822fe2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.203Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-70623d97a6d4808aebceccb7: done: 0.071117 sec","job_status":"done","queue_duration_s":1.228452,"scheduling_latency_s":1.228521,"redis_calls":12,"redis_duration_s":0.004752,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001487,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002318,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000551187,"duration_s":0.071117,"completed_at":"2026-06-29T21:56:10.502Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453vm-test-run-nixbot-gitlab> gitlab # [ 157.581212] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.573Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"51e3ac4faf3b4dddc10c6559","created_at":"2026-06-29T21:56:09.246Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d6c8e60a486677418eb5b6e6170c58a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.257Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-51e3ac4faf3b4dddc10c6559: start","job_status":"start","queue_duration_s":1.316368,"scheduling_latency_s":1.316438}1454vm-test-run-nixbot-gitlab> gitlab # [ 157.630397] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"51e3ac4faf3b4dddc10c6559","created_at":"2026-06-29T21:56:09.246Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d6c8e60a486677418eb5b6e6170c58a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.257Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-51e3ac4faf3b4dddc10c6559: done: 0.049253 sec","job_status":"done","queue_duration_s":1.316368,"scheduling_latency_s":1.316438,"redis_calls":10,"redis_duration_s":0.006928999999999999,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001137,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004247,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001545,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000980571,"duration_s":0.049253,"completed_at":"2026-06-29T21:56:10.622Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001217,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455vm-test-run-nixbot-gitlab> gitlab # [ 157.701535] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b75ea9949e01f28a0a485d2e","created_at":"2026-06-29T21:56:09.274Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca38ddd6405a4de429866a267edbd57a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.283Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b75ea9949e01f28a0a485d2e: start","job_status":"start","queue_duration_s":1.409401,"scheduling_latency_s":1.409491}1456vm-test-run-nixbot-gitlab> gitlab # [ 157.778738] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.770Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b75ea9949e01f28a0a485d2e","created_at":"2026-06-29T21:56:09.274Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca38ddd6405a4de429866a267edbd57a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.283Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b75ea9949e01f28a0a485d2e: done: 0.077676 sec","job_status":"done","queue_duration_s":1.409401,"scheduling_latency_s":1.409491,"redis_calls":12,"redis_duration_s":0.006469000000000001,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000745,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002393,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001086451,"exclusive_lock_hold_duration_s":0.009706540999999999,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.077676,"completed_at":"2026-06-29T21:56:10.770Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005905,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457vm-test-run-nixbot-gitlab> gitlab # [ 157.784903] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a70f91fea9579db5e5e7a071","created_at":"2026-06-29T21:56:09.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13ea21c41f1804baa88cdd758c3c53e2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.307Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a70f91fea9579db5e5e7a071: start","job_status":"start","queue_duration_s":1.470198,"scheduling_latency_s":1.470251}1458vm-test-run-nixbot-gitlab> gitlab # [ 157.841954] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a70f91fea9579db5e5e7a071","created_at":"2026-06-29T21:56:09.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13ea21c41f1804baa88cdd758c3c53e2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.307Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a70f91fea9579db5e5e7a071: done: 0.056862 sec","job_status":"done","queue_duration_s":1.470198,"scheduling_latency_s":1.470251,"redis_calls":12,"redis_duration_s":0.008822,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001008,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003772,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001206299,"exclusive_lock_hold_duration_s":0.010694376,"duration_s":0.056862,"completed_at":"2026-06-29T21:56:10.834Z","load_balancing_strategy":"primary","db_duration_s":0.006617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459vm-test-run-nixbot-gitlab> gitlab # [ 157.871265] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8d51c7c753797650eae45693","created_at":"2026-06-29T21:56:09.321Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a9a58b929e2c6352e173046a74dd4e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.330Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8d51c7c753797650eae45693: start","job_status":"start","queue_duration_s":1.510483,"scheduling_latency_s":1.510527}1460vm-test-run-nixbot-gitlab> gitlab # [ 157.900983] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.893Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8d51c7c753797650eae45693","created_at":"2026-06-29T21:56:09.321Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a9a58b929e2c6352e173046a74dd4e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.330Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8d51c7c753797650eae45693: done: 0.052589 sec","job_status":"done","queue_duration_s":1.510483,"scheduling_latency_s":1.510527,"redis_calls":14,"redis_duration_s":0.009089,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002363,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003926,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0028,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000893409,"exclusive_lock_hold_duration_s":0.016108471000000003,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.052589,"completed_at":"2026-06-29T21:56:10.893Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013333,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461vm-test-run-nixbot-gitlab> gitlab # [ 158.000419] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:10.899Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"ec0de226333c45668e7e1481","created_at":"2026-06-29T21:56:09.342Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c701e92165c80789ed24faf6c0fc998b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.349Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-ec0de226333c45668e7e1481: start","job_status":"start","queue_duration_s":1.549966,"scheduling_latency_s":1.54999}1462vm-test-run-nixbot-gitlab> gitlab # [ 158.029367] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.021Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"ec0de226333c45668e7e1481","created_at":"2026-06-29T21:56:09.342Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c701e92165c80789ed24faf6c0fc998b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.349Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-ec0de226333c45668e7e1481: done: 0.121963 sec","job_status":"done","queue_duration_s":1.549966,"scheduling_latency_s":1.54999,"redis_calls":12,"redis_duration_s":0.008102,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001147,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003615,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00334,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001343188,"exclusive_lock_hold_duration_s":0.096467288,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.121963,"completed_at":"2026-06-29T21:56:11.021Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013383,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1463vm-test-run-nixbot-gitlab> gitlab # [ 158.061565] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0477704f3026d0a0238e5a7b","created_at":"2026-06-29T21:56:09.365Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4de4d0b08615731dccd112fe850a9b7f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.372Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0477704f3026d0a0238e5a7b: start","job_status":"start","queue_duration_s":1.651529,"scheduling_latency_s":1.651553}1464vm-test-run-nixbot-gitlab> gitlab # [ 158.147200] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.037Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1465vm-test-run-nixbot-gitlab> gitlab # [ 158.148582] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.043Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0477704f3026d0a0238e5a7b","created_at":"2026-06-29T21:56:09.365Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4de4d0b08615731dccd112fe850a9b7f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.372Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0477704f3026d0a0238e5a7b: done: 0.019175 sec","job_status":"done","queue_duration_s":1.651529,"scheduling_latency_s":1.651553,"redis_calls":10,"redis_duration_s":0.002405,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000348927,"duration_s":0.019175,"completed_at":"2026-06-29T21:56:11.043Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001252,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466vm-test-run-nixbot-gitlab> gitlab # [ 158.157699] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.045Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8976381c621e2b04c0f1aed9","created_at":"2026-06-29T21:56:09.381Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"520f1269f46cf0be7cabc1c73b097e17","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T21:56:09.402Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8976381c621e2b04c0f1aed9: start","job_status":"start","queue_duration_s":1.642836,"scheduling_latency_s":1.642856}1467vm-test-run-nixbot-gitlab> gitlab # [ 158.179742] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8976381c621e2b04c0f1aed9","created_at":"2026-06-29T21:56:09.381Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"520f1269f46cf0be7cabc1c73b097e17","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T21:56:09.402Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8976381c621e2b04c0f1aed9: done: 0.028723 sec","job_status":"done","queue_duration_s":1.642836,"scheduling_latency_s":1.642856,"redis_calls":10,"redis_duration_s":0.00227,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001395,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000344457,"duration_s":0.028723,"completed_at":"2026-06-29T21:56:11.073Z","load_balancing_strategy":"primary","db_duration_s":0.005197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468vm-test-run-nixbot-gitlab> gitlab # [ 158.246463] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5e28b61d303ed5679a53dd82","created_at":"2026-06-29T21:56:09.415Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"224fcadb060758057860fa9a44192bda","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T21:56:09.426Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5e28b61d303ed5679a53dd82: start","job_status":"start","queue_duration_s":1.649184,"scheduling_latency_s":1.649205}1469vm-test-run-nixbot-gitlab> gitlab # [ 158.268301] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.086Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1470vm-test-run-nixbot-gitlab> gitlab # [ 158.274806] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5e28b61d303ed5679a53dd82","created_at":"2026-06-29T21:56:09.415Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"224fcadb060758057860fa9a44192bda","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T21:56:09.426Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5e28b61d303ed5679a53dd82: done: 0.01698 sec","job_status":"done","queue_duration_s":1.649184,"scheduling_latency_s":1.649205,"redis_calls":10,"redis_duration_s":0.002321,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000481,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001392,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355911,"duration_s":0.01698,"completed_at":"2026-06-29T21:56:11.092Z","load_balancing_strategy":"primary","db_duration_s":0.000397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471vm-test-run-nixbot-gitlab> gitlab # [ 158.342279] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.094Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8a824818bcb2d0af3675a657","created_at":"2026-06-29T21:56:09.439Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fffb26a02f728a6513b328c825249d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.445Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8a824818bcb2d0af3675a657: start","job_status":"start","queue_duration_s":1.648471,"scheduling_latency_s":1.6485}1472vm-test-run-nixbot-gitlab> gitlab # [ 158.366167] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.120Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8a824818bcb2d0af3675a657","created_at":"2026-06-29T21:56:09.439Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fffb26a02f728a6513b328c825249d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.445Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8a824818bcb2d0af3675a657: done: 0.026556 sec","job_status":"done","queue_duration_s":1.648471,"scheduling_latency_s":1.6485,"redis_calls":10,"redis_duration_s":0.0046300000000000004,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000965,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002663,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000493638,"duration_s":0.026556,"completed_at":"2026-06-29T21:56:11.120Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473vm-test-run-nixbot-gitlab> gitlab # [ 158.444243] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.122Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bc5ecd265e08cff5de26b07b","created_at":"2026-06-29T21:56:09.450Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5d15f3bb82baecce77d6cdb6e337772","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.457Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bc5ecd265e08cff5de26b07b: start","job_status":"start","queue_duration_s":1.665923,"scheduling_latency_s":1.665944}1474vm-test-run-nixbot-gitlab> gitlab # [ 158.457186] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.135Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1475vm-test-run-nixbot-gitlab> gitlab # [ 158.463167] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bc5ecd265e08cff5de26b07b","created_at":"2026-06-29T21:56:09.450Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5d15f3bb82baecce77d6cdb6e337772","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.457Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bc5ecd265e08cff5de26b07b: done: 0.022264 sec","job_status":"done","queue_duration_s":1.665923,"scheduling_latency_s":1.665944,"redis_calls":10,"redis_duration_s":0.006410000000000001,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00084,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000804,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000526044,"duration_s":0.022264,"completed_at":"2026-06-29T21:56:11.145Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476vm-test-run-nixbot-gitlab> gitlab # [ 158.533052] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2e4d185aa455e33137e120c5","created_at":"2026-06-29T21:56:09.464Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb43bcf9c04457334c2679e289752250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.471Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2e4d185aa455e33137e120c5: start","job_status":"start","queue_duration_s":1.675482,"scheduling_latency_s":1.675503}1477vm-test-run-nixbot-gitlab> gitlab # [ 158.557303] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.178Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2e4d185aa455e33137e120c5","created_at":"2026-06-29T21:56:09.464Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb43bcf9c04457334c2679e289752250","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.471Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2e4d185aa455e33137e120c5: done: 0.032019 sec","job_status":"done","queue_duration_s":1.675482,"scheduling_latency_s":1.675503,"redis_calls":12,"redis_duration_s":0.0029590000000000003,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000526603,"exclusive_lock_hold_duration_s":0.008484598,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.032019,"completed_at":"2026-06-29T21:56:11.178Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009201,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478vm-test-run-nixbot-gitlab> gitlab # [ 158.631358] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.180Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f2da4d5d011c996ed283f199","created_at":"2026-06-29T21:56:09.477Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3672e5146cfd25174b52ffa5c13ff0d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.484Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f2da4d5d011c996ed283f199: start","job_status":"start","queue_duration_s":1.69628,"scheduling_latency_s":1.696302}1479vm-test-run-nixbot-gitlab> gitlab # [ 158.654246] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.208Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f2da4d5d011c996ed283f199","created_at":"2026-06-29T21:56:09.477Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3672e5146cfd25174b52ffa5c13ff0d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.484Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f2da4d5d011c996ed283f199: done: 0.027798 sec","job_status":"done","queue_duration_s":1.69628,"scheduling_latency_s":1.696302,"redis_calls":10,"redis_duration_s":0.0024360000000000002,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000486,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001502,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000406476,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.027798,"completed_at":"2026-06-29T21:56:11.208Z","load_balancing_strategy":"primary","db_duration_s":0.008885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480vm-test-run-nixbot-gitlab> gitlab # [ 158.740882] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.209Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2dedb77980681ea0b3be5990","created_at":"2026-06-29T21:56:09.491Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73c26353659d054e0af7ed0cf22d987f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.497Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2dedb77980681ea0b3be5990: start","job_status":"start","queue_duration_s":1.711892,"scheduling_latency_s":1.711912}1481vm-test-run-nixbot-gitlab> gitlab # [ 158.744360] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2dedb77980681ea0b3be5990","created_at":"2026-06-29T21:56:09.491Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73c26353659d054e0af7ed0cf22d987f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.497Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2dedb77980681ea0b3be5990: done: 0.419462 sec","job_status":"done","queue_duration_s":1.711892,"scheduling_latency_s":1.711912,"redis_calls":574,"redis_duration_s":0.13395,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000486,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001547,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.131917,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.3112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000374628,"duration_s":0.419462,"completed_at":"2026-06-29T21:56:11.629Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482vm-test-run-nixbot-gitlab> gitlab # [ 158.812911] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.630Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f406aea99a3494b1c1c0816a","created_at":"2026-06-29T21:56:09.502Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.509Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f406aea99a3494b1c1c0816a: start","job_status":"start","queue_duration_s":2.121498,"scheduling_latency_s":2.121522}1483vm-test-run-nixbot-gitlab> gitlab # [ 158.886389] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f406aea99a3494b1c1c0816a","created_at":"2026-06-29T21:56:09.502Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.509Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f406aea99a3494b1c1c0816a: done: 0.096742 sec","job_status":"done","queue_duration_s":2.121498,"scheduling_latency_s":2.121522,"redis_calls":71,"redis_duration_s":0.017008,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002293,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000732,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013259,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000724,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000391391,"duration_s":0.096742,"completed_at":"2026-06-29T21:56:11.727Z","load_balancing_strategy":"primary","db_duration_s":0.002156,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484vm-test-run-nixbot-gitlab> gitlab # [ 158.902237] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.729Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"0cc6498d828b803a7586878d","created_at":"2026-06-29T21:56:09.517Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"ef76a3d252579a385563577dbdd256a1","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.524Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-0cc6498d828b803a7586878d: start","job_status":"start","queue_duration_s":2.204351,"scheduling_latency_s":2.204374}1485vm-test-run-nixbot-gitlab> gitlab # [ 158.925256] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.749Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"0cc6498d828b803a7586878d","created_at":"2026-06-29T21:56:09.517Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"ef76a3d252579a385563577dbdd256a1","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.524Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-0cc6498d828b803a7586878d: done: 0.020566 sec","job_status":"done","queue_duration_s":2.204351,"scheduling_latency_s":2.204374,"redis_calls":10,"redis_duration_s":0.004595999999999999,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001465,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002701,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034306,"duration_s":0.020566,"completed_at":"2026-06-29T21:56:11.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000612,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486vm-test-run-nixbot-gitlab> gitlab # [ 158.993573] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.751Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"12d422cf849b80c9e845bdfa","created_at":"2026-06-29T21:56:09.529Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"957079a4069091df21665ad9ac1589ae","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.535Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-12d422cf849b80c9e845bdfa: start","job_status":"start","queue_duration_s":2.215499,"scheduling_latency_s":2.215519}1487vm-test-run-nixbot-gitlab> gitlab # [ 159.015526] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.769Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"12d422cf849b80c9e845bdfa","created_at":"2026-06-29T21:56:09.529Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"957079a4069091df21665ad9ac1589ae","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.535Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-12d422cf849b80c9e845bdfa: done: 0.018061 sec","job_status":"done","queue_duration_s":2.215499,"scheduling_latency_s":2.215519,"redis_calls":12,"redis_duration_s":0.003016,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000579,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001557,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039167,"duration_s":0.018061,"completed_at":"2026-06-29T21:56:11.769Z","load_balancing_strategy":"primary","db_duration_s":0.000333,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488vm-test-run-nixbot-gitlab> gitlab # [ 159.085235] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.770Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3c4f8e47eff5ace591b68109","created_at":"2026-06-29T21:56:09.540Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a4ec188241caddb33c350be4f982a89","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.547Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3c4f8e47eff5ace591b68109: start","job_status":"start","queue_duration_s":2.223406,"scheduling_latency_s":2.223429}1489vm-test-run-nixbot-gitlab> gitlab # [ 159.108992] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3c4f8e47eff5ace591b68109","created_at":"2026-06-29T21:56:09.540Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a4ec188241caddb33c350be4f982a89","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.547Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3c4f8e47eff5ace591b68109: done: 0.026889 sec","job_status":"done","queue_duration_s":2.223406,"scheduling_latency_s":2.223429,"redis_calls":13,"redis_duration_s":0.003287,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000596,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001648,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008191265,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.026889,"completed_at":"2026-06-29T21:56:11.797Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002925,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1490vm-test-run-nixbot-gitlab> gitlab # [ 159.184100] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8cd0d9e57903d3ec9a06d715","created_at":"2026-06-29T21:56:09.558Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63943448a11dc36383fc929909d65e2d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.565Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8cd0d9e57903d3ec9a06d715: start","job_status":"start","queue_duration_s":2.233104,"scheduling_latency_s":2.233124}1491vm-test-run-nixbot-gitlab> gitlab # [ 159.202216] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.810Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1492vm-test-run-nixbot-gitlab> gitlab # [ 159.208815] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8cd0d9e57903d3ec9a06d715","created_at":"2026-06-29T21:56:09.558Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63943448a11dc36383fc929909d65e2d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.565Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8cd0d9e57903d3ec9a06d715: done: 0.017224 sec","job_status":"done","queue_duration_s":2.233104,"scheduling_latency_s":2.233124,"redis_calls":10,"redis_duration_s":0.002406,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000486,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035619,"duration_s":0.017224,"completed_at":"2026-06-29T21:56:11.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493vm-test-run-nixbot-gitlab> gitlab # [ 159.329136] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.817Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8b586f685a95d761b97100d2","created_at":"2026-06-29T21:56:09.575Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.582Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8b586f685a95d761b97100d2: start","job_status":"start","queue_duration_s":2.235473,"scheduling_latency_s":2.235494}1494vm-test-run-nixbot-gitlab> gitlab # [ 159.330849] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.846Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8b586f685a95d761b97100d2","created_at":"2026-06-29T21:56:09.575Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.582Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8b586f685a95d761b97100d2: done: 0.028587 sec","job_status":"done","queue_duration_s":2.235473,"scheduling_latency_s":2.235494,"redis_calls":19,"redis_duration_s":0.004223,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000953,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000329,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002255,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000686,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355911,"duration_s":0.028587,"completed_at":"2026-06-29T21:56:11.846Z","load_balancing_strategy":"primary","db_duration_s":0.001527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495vm-test-run-nixbot-gitlab> gitlab # [ 159.415360] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.899Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"2f0bc9200447989b87a69fce","created_at":"2026-06-29T21:56:09.598Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c8897bd206359e89c7cc9a4eb0215ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.606Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-2f0bc9200447989b87a69fce: start","job_status":"start","queue_duration_s":2.293295,"scheduling_latency_s":2.293328}1496vm-test-run-nixbot-gitlab> gitlab # [ 159.416757] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.913Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1497vm-test-run-nixbot-gitlab> gitlab # [ 159.421655] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.919Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"2f0bc9200447989b87a69fce","created_at":"2026-06-29T21:56:09.598Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c8897bd206359e89c7cc9a4eb0215ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.606Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-2f0bc9200447989b87a69fce: done: 0.019142 sec","job_status":"done","queue_duration_s":2.293295,"scheduling_latency_s":2.293328,"redis_calls":10,"redis_duration_s":0.002452,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001494,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036066,"duration_s":0.019142,"completed_at":"2026-06-29T21:56:11.918Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498vm-test-run-nixbot-gitlab> gitlab # [ 159.463242] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.920Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"0c262519436608d60933f818","created_at":"2026-06-29T21:56:09.616Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a7615bb6a01d14697aab59032301e87","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.623Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-0c262519436608d60933f818: start","job_status":"start","queue_duration_s":2.29673,"scheduling_latency_s":2.296751}1499vm-test-run-nixbot-gitlab> gitlab # [ 159.487212] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"0c262519436608d60933f818","created_at":"2026-06-29T21:56:09.616Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a7615bb6a01d14697aab59032301e87","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.623Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-0c262519436608d60933f818: done: 0.020091 sec","job_status":"done","queue_duration_s":2.29673,"scheduling_latency_s":2.296751,"redis_calls":12,"redis_duration_s":0.003269,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001557,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001276,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00044,"exclusive_lock_hold_duration_s":0.003421943,"duration_s":0.020091,"completed_at":"2026-06-29T21:56:11.940Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001832,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500vm-test-run-nixbot-gitlab> gitlab # [ 159.621586] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.941Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ExpireBuildArtifactsWorker","jid":"9147384eec19b8bd2a58470b","created_at":"2026-06-29T21:56:09.641Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cdc3129ec6b1a57f14e44120d5298c8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:7be9ea93da0bda6381b334cf2932692b41842a57ea0aeac11899369225b9bb7e","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.649Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ExpireBuildArtifactsWorker JID-9147384eec19b8bd2a58470b: start","job_status":"start","queue_duration_s":2.292059,"scheduling_latency_s":2.292081}1501vm-test-run-nixbot-gitlab> gitlab # [ 159.623286] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.957Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ExpireBuildArtifactsWorker","jid":"9147384eec19b8bd2a58470b","created_at":"2026-06-29T21:56:09.641Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cdc3129ec6b1a57f14e44120d5298c8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:7be9ea93da0bda6381b334cf2932692b41842a57ea0aeac11899369225b9bb7e","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.649Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ExpireBuildArtifactsWorker JID-9147384eec19b8bd2a58470b: done: 0.015927 sec","job_status":"done","queue_duration_s":2.292059,"scheduling_latency_s":2.292081,"redis_calls":10,"redis_duration_s":0.002399,"redis_read_bytes":9,"redis_write_bytes":1035,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001377,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000377422,"duration_s":0.015927,"completed_at":"2026-06-29T21:56:11.957Z","load_balancing_strategy":"primary","db_duration_s":0.000368,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502vm-test-run-nixbot-gitlab> gitlab # [ 159.705214] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.959Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ea801095452b71500756a698","created_at":"2026-06-29T21:56:09.655Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cde0bb28e3a7c7aae49cd8594599ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.662Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ea801095452b71500756a698: start","job_status":"start","queue_duration_s":2.296717,"scheduling_latency_s":2.296737}1503vm-test-run-nixbot-gitlab> gitlab # [ 159.706511] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ea801095452b71500756a698","created_at":"2026-06-29T21:56:09.655Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cde0bb28e3a7c7aae49cd8594599ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:09.662Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ea801095452b71500756a698: done: 0.018325 sec","job_status":"done","queue_duration_s":2.296717,"scheduling_latency_s":2.296737,"redis_calls":10,"redis_duration_s":0.002372,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000534,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001383,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000336915,"duration_s":0.018325,"completed_at":"2026-06-29T21:56:11.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001796,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504vm-test-run-nixbot-gitlab> gitlab # [ 159.730398] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:11.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1788fe45e49ea6d3370317e9","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1788fe45e49ea6d3370317e9: start","job_status":"start","queue_duration_s":0.259994,"scheduling_latency_s":0.260015}1505vm-test-run-nixbot-gitlab> gitlab # [ 159.750562] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1788fe45e49ea6d3370317e9","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1788fe45e49ea6d3370317e9: done: 0.02157 sec","job_status":"done","queue_duration_s":0.259994,"scheduling_latency_s":0.260015,"redis_calls":16,"redis_duration_s":0.004243,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002196,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000385523,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02157,"completed_at":"2026-06-29T21:56:12.000Z","load_balancing_strategy":"primary","db_duration_s":0.000389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506vm-test-run-nixbot-gitlab> gitlab # [ 159.817078] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27afd6adfdff6878aeb763d6","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27afd6adfdff6878aeb763d6: start","job_status":"start","queue_duration_s":0.282952,"scheduling_latency_s":0.282973}1507vm-test-run-nixbot-gitlab> gitlab # [ 159.837071] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27afd6adfdff6878aeb763d6","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27afd6adfdff6878aeb763d6: done: 0.015661 sec","job_status":"done","queue_duration_s":0.282952,"scheduling_latency_s":0.282973,"redis_calls":13,"redis_duration_s":0.003152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015661,"completed_at":"2026-06-29T21:56:12.017Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508vm-test-run-nixbot-gitlab> gitlab # [ 159.913410] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad397e7768d76db94bc749e9","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad397e7768d76db94bc749e9: start","job_status":"start","queue_duration_s":0.30029,"scheduling_latency_s":0.300312}1509vm-test-run-nixbot-gitlab> gitlab # [ 159.919282] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad397e7768d76db94bc749e9","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad397e7768d76db94bc749e9: done: 0.015408 sec","job_status":"done","queue_duration_s":0.30029,"scheduling_latency_s":0.300312,"redis_calls":13,"redis_duration_s":0.00298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015408,"completed_at":"2026-06-29T21:56:12.034Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510vm-test-run-nixbot-gitlab> gitlab # [ 159.982275] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"086b2cca6fda90cec5b89488","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-086b2cca6fda90cec5b89488: start","job_status":"start","queue_duration_s":0.316975,"scheduling_latency_s":0.316997}1511vm-test-run-nixbot-gitlab> gitlab # [ 160.058392] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"086b2cca6fda90cec5b89488","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-086b2cca6fda90cec5b89488: done: 0.015461 sec","job_status":"done","queue_duration_s":0.316975,"scheduling_latency_s":0.316997,"redis_calls":13,"redis_duration_s":0.003173,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002285,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015461,"completed_at":"2026-06-29T21:56:12.051Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512vm-test-run-nixbot-gitlab> gitlab # [ 160.064291] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd303c919c5a6ae74a3133bb","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd303c919c5a6ae74a3133bb: start","job_status":"start","queue_duration_s":0.33372,"scheduling_latency_s":0.33374}1513vm-test-run-nixbot-gitlab> gitlab # [ 160.084578] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd303c919c5a6ae74a3133bb","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd303c919c5a6ae74a3133bb: done: 0.016044 sec","job_status":"done","queue_duration_s":0.33372,"scheduling_latency_s":0.33374,"redis_calls":13,"redis_duration_s":0.003313,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002382,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016044,"completed_at":"2026-06-29T21:56:12.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514vm-test-run-nixbot-gitlab> gitlab # [ 160.205514] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db5447246b6d84d5f1b0d15a","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db5447246b6d84d5f1b0d15a: start","job_status":"start","queue_duration_s":0.351212,"scheduling_latency_s":0.351235}1515vm-test-run-nixbot-gitlab> gitlab # [ 160.206780] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db5447246b6d84d5f1b0d15a","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db5447246b6d84d5f1b0d15a: done: 0.015476 sec","job_status":"done","queue_duration_s":0.351212,"scheduling_latency_s":0.351235,"redis_calls":13,"redis_duration_s":0.0029899999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015476,"completed_at":"2026-06-29T21:56:12.085Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516vm-test-run-nixbot-gitlab> gitlab # [ 160.231046] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f07943263d57afc3eab53ac","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f07943263d57afc3eab53ac: start","job_status":"start","queue_duration_s":0.368293,"scheduling_latency_s":0.368314}1517vm-test-run-nixbot-gitlab> gitlab # [ 160.251527] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f07943263d57afc3eab53ac","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f07943263d57afc3eab53ac: done: 0.015703 sec","job_status":"done","queue_duration_s":0.368293,"scheduling_latency_s":0.368314,"redis_calls":13,"redis_duration_s":0.003117,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015703,"completed_at":"2026-06-29T21:56:12.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518vm-test-run-nixbot-gitlab> gitlab # [ 160.315323] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cc93e389b5ee1d0772d384e","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cc93e389b5ee1d0772d384e: start","job_status":"start","queue_duration_s":0.385715,"scheduling_latency_s":0.385738}1519vm-test-run-nixbot-gitlab> gitlab # [ 160.335229] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cc93e389b5ee1d0772d384e","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cc93e389b5ee1d0772d384e: done: 0.015623 sec","job_status":"done","queue_duration_s":0.385715,"scheduling_latency_s":0.385738,"redis_calls":13,"redis_duration_s":0.003125,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001015,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015623,"completed_at":"2026-06-29T21:56:12.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520vm-test-run-nixbot-gitlab> gitlab # [ 160.399232] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9859321273e99c6e44028162","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9859321273e99c6e44028162: start","job_status":"start","queue_duration_s":0.40267,"scheduling_latency_s":0.402691}1521vm-test-run-nixbot-gitlab> gitlab # [ 160.419354] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9859321273e99c6e44028162","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9859321273e99c6e44028162: done: 0.015325 sec","job_status":"done","queue_duration_s":0.40267,"scheduling_latency_s":0.402691,"redis_calls":13,"redis_duration_s":0.002946,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015325,"completed_at":"2026-06-29T21:56:12.137Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522vm-test-run-nixbot-gitlab> gitlab # [ 160.499459] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5799a87a57f608e1bda3544","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5799a87a57f608e1bda3544: start","job_status":"start","queue_duration_s":0.419453,"scheduling_latency_s":0.419474}1523vm-test-run-nixbot-gitlab> gitlab # [ 160.501113] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5799a87a57f608e1bda3544","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5799a87a57f608e1bda3544: done: 0.015951 sec","job_status":"done","queue_duration_s":0.419453,"scheduling_latency_s":0.419474,"redis_calls":13,"redis_duration_s":0.002994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015951,"completed_at":"2026-06-29T21:56:12.154Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524vm-test-run-nixbot-gitlab> gitlab # [ 160.563552] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e6ffa71a2fc628dda7973e8","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e6ffa71a2fc628dda7973e8: start","job_status":"start","queue_duration_s":0.436781,"scheduling_latency_s":0.436802}1525vm-test-run-nixbot-gitlab> gitlab # [ 160.583909] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e6ffa71a2fc628dda7973e8","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e6ffa71a2fc628dda7973e8: done: 0.01541 sec","job_status":"done","queue_duration_s":0.436781,"scheduling_latency_s":0.436802,"redis_calls":13,"redis_duration_s":0.00294,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01541,"completed_at":"2026-06-29T21:56:12.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526vm-test-run-nixbot-gitlab> gitlab # [ 160.647246] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fb605aa34dae1aa6a95c3bd","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fb605aa34dae1aa6a95c3bd: start","job_status":"start","queue_duration_s":0.45347,"scheduling_latency_s":0.453491}1527vm-test-run-nixbot-gitlab> gitlab # [ 160.667367] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fb605aa34dae1aa6a95c3bd","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fb605aa34dae1aa6a95c3bd: done: 0.022155 sec","job_status":"done","queue_duration_s":0.45347,"scheduling_latency_s":0.453491,"redis_calls":13,"redis_duration_s":0.009701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022155,"completed_at":"2026-06-29T21:56:12.194Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528vm-test-run-nixbot-gitlab> gitlab # [ 160.731333] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa07ef54322c79b95baba682","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa07ef54322c79b95baba682: start","job_status":"start","queue_duration_s":0.47693,"scheduling_latency_s":0.476953}1529vm-test-run-nixbot-gitlab> gitlab # [ 160.808756] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa07ef54322c79b95baba682","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa07ef54322c79b95baba682: done: 0.015967 sec","job_status":"done","queue_duration_s":0.47693,"scheduling_latency_s":0.476953,"redis_calls":13,"redis_duration_s":0.003099,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015967,"completed_at":"2026-06-29T21:56:12.212Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530vm-test-run-nixbot-gitlab> gitlab # [ 160.811868] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f06e89d076ad3092be96b01e","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f06e89d076ad3092be96b01e: start","job_status":"start","queue_duration_s":0.494622,"scheduling_latency_s":0.494643}1531vm-test-run-nixbot-gitlab> gitlab # [ 160.832105] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f06e89d076ad3092be96b01e","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f06e89d076ad3092be96b01e: done: 0.015409 sec","job_status":"done","queue_duration_s":0.494622,"scheduling_latency_s":0.494643,"redis_calls":13,"redis_duration_s":0.0030169999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015409,"completed_at":"2026-06-29T21:56:12.229Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532vm-test-run-nixbot-gitlab> gitlab # [ 160.937755] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0acce21ddebaaac52c4e45d","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0acce21ddebaaac52c4e45d: start","job_status":"start","queue_duration_s":0.511375,"scheduling_latency_s":0.511396}1533vm-test-run-nixbot-gitlab> gitlab # [ 160.939477] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0acce21ddebaaac52c4e45d","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0acce21ddebaaac52c4e45d: done: 0.01534 sec","job_status":"done","queue_duration_s":0.511375,"scheduling_latency_s":0.511396,"redis_calls":13,"redis_duration_s":0.0033550000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002416,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01534,"completed_at":"2026-06-29T21:56:12.245Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534vm-test-run-nixbot-gitlab> gitlab # [ 160.978922] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2119c99e674b35877e3051cf","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2119c99e674b35877e3051cf: start","job_status":"start","queue_duration_s":0.528094,"scheduling_latency_s":0.528115}1535vm-test-run-nixbot-gitlab> gitlab # [ 160.998803] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2119c99e674b35877e3051cf","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2119c99e674b35877e3051cf: done: 0.017118 sec","job_status":"done","queue_duration_s":0.528094,"scheduling_latency_s":0.528115,"redis_calls":13,"redis_duration_s":0.004738,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003754,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017118,"completed_at":"2026-06-29T21:56:12.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536vm-test-run-nixbot-gitlab> gitlab # [ 161.084506] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f27bf68e784f455a5933481","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f27bf68e784f455a5933481: start","job_status":"start","queue_duration_s":0.546595,"scheduling_latency_s":0.546618}1537vm-test-run-nixbot-gitlab> gitlab # [ 161.086187] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f27bf68e784f455a5933481","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f27bf68e784f455a5933481: done: 0.017359 sec","job_status":"done","queue_duration_s":0.546595,"scheduling_latency_s":0.546618,"redis_calls":13,"redis_duration_s":0.003076,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017359,"completed_at":"2026-06-29T21:56:12.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538vm-test-run-nixbot-gitlab> gitlab # [ 161.144174] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ebbd5b871fce420e46998f5","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ebbd5b871fce420e46998f5: start","job_status":"start","queue_duration_s":0.565859,"scheduling_latency_s":0.565887}1539vm-test-run-nixbot-gitlab> gitlab # [ 161.229805] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ebbd5b871fce420e46998f5","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ebbd5b871fce420e46998f5: done: 0.022675 sec","job_status":"done","queue_duration_s":0.565859,"scheduling_latency_s":0.565887,"redis_calls":13,"redis_duration_s":0.009258,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003839,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005419,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022675,"completed_at":"2026-06-29T21:56:12.307Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540vm-test-run-nixbot-gitlab> gitlab # [ 161.231932] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58493285182b6603757dbf7f","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58493285182b6603757dbf7f: start","job_status":"start","queue_duration_s":0.59133,"scheduling_latency_s":0.591361}1541vm-test-run-nixbot-gitlab> gitlab # [ 161.245296] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58493285182b6603757dbf7f","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58493285182b6603757dbf7f: done: 0.019928 sec","job_status":"done","queue_duration_s":0.59133,"scheduling_latency_s":0.591361,"redis_calls":13,"redis_duration_s":0.005837,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000975,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019928,"completed_at":"2026-06-29T21:56:12.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542vm-test-run-nixbot-gitlab> gitlab # [ 161.374621] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9cd74cd8a76e1ff525e535d","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9cd74cd8a76e1ff525e535d: start","job_status":"start","queue_duration_s":0.612548,"scheduling_latency_s":0.61257}1543vm-test-run-nixbot-gitlab> gitlab # [ 161.375771] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9cd74cd8a76e1ff525e535d","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9cd74cd8a76e1ff525e535d: done: 0.015366 sec","job_status":"done","queue_duration_s":0.612548,"scheduling_latency_s":0.61257,"redis_calls":13,"redis_duration_s":0.002926,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015366,"completed_at":"2026-06-29T21:56:12.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544vm-test-run-nixbot-gitlab> gitlab # [ 161.446659] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e862f28b5e4b64e94109eae","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e862f28b5e4b64e94109eae: start","job_status":"start","queue_duration_s":0.629292,"scheduling_latency_s":0.629314}1545vm-test-run-nixbot-gitlab> gitlab # [ 161.448707] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e862f28b5e4b64e94109eae","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e862f28b5e4b64e94109eae: done: 0.015486 sec","job_status":"done","queue_duration_s":0.629292,"scheduling_latency_s":0.629314,"redis_calls":13,"redis_duration_s":0.002872,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015486,"completed_at":"2026-06-29T21:56:12.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546vm-test-run-nixbot-gitlab> gitlab # [ 161.473625] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf62e98077622fc015d73a72","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf62e98077622fc015d73a72: start","job_status":"start","queue_duration_s":0.64613,"scheduling_latency_s":0.646152}1547vm-test-run-nixbot-gitlab> gitlab # [ 161.494301] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf62e98077622fc015d73a72","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf62e98077622fc015d73a72: done: 0.019355 sec","job_status":"done","queue_duration_s":0.64613,"scheduling_latency_s":0.646152,"redis_calls":13,"redis_duration_s":0.002892,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019355,"completed_at":"2026-06-29T21:56:12.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548vm-test-run-nixbot-gitlab> gitlab # [ 161.558168] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"475a06cd2b5872dfad3622d3","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-475a06cd2b5872dfad3622d3: start","job_status":"start","queue_duration_s":0.666754,"scheduling_latency_s":0.666774}1549vm-test-run-nixbot-gitlab> gitlab # [ 161.578441] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"475a06cd2b5872dfad3622d3","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-475a06cd2b5872dfad3622d3: done: 0.015698 sec","job_status":"done","queue_duration_s":0.666754,"scheduling_latency_s":0.666774,"redis_calls":13,"redis_duration_s":0.003352,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001337,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015698,"completed_at":"2026-06-29T21:56:12.401Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550vm-test-run-nixbot-gitlab> gitlab # [ 161.641853] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe6bbc2e51490a0dbd508e1b","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe6bbc2e51490a0dbd508e1b: start","job_status":"start","queue_duration_s":0.683718,"scheduling_latency_s":0.68374}1551vm-test-run-nixbot-gitlab> gitlab # [ 161.661970] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe6bbc2e51490a0dbd508e1b","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe6bbc2e51490a0dbd508e1b: done: 0.028734 sec","job_status":"done","queue_duration_s":0.683718,"scheduling_latency_s":0.68374,"redis_calls":13,"redis_duration_s":0.014025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012442,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028734,"completed_at":"2026-06-29T21:56:12.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552vm-test-run-nixbot-gitlab> gitlab # [ 161.725330] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e55c109da3ce0ca90829433e","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e55c109da3ce0ca90829433e: start","job_status":"start","queue_duration_s":0.714749,"scheduling_latency_s":0.714778}1553vm-test-run-nixbot-gitlab> gitlab # [ 161.745177] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e55c109da3ce0ca90829433e","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e55c109da3ce0ca90829433e: done: 0.017023 sec","job_status":"done","queue_duration_s":0.714749,"scheduling_latency_s":0.714778,"redis_calls":13,"redis_duration_s":0.003245,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001108,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017023,"completed_at":"2026-06-29T21:56:12.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554vm-test-run-nixbot-gitlab> gitlab # [ 161.816810] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b49f8f6a8015552b1e4130a","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b49f8f6a8015552b1e4130a: start","job_status":"start","queue_duration_s":0.733029,"scheduling_latency_s":0.733102}1555vm-test-run-nixbot-gitlab> gitlab # [ 161.827306] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b49f8f6a8015552b1e4130a","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b49f8f6a8015552b1e4130a: done: 0.015702 sec","job_status":"done","queue_duration_s":0.733029,"scheduling_latency_s":0.733102,"redis_calls":13,"redis_duration_s":0.003029,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015702,"completed_at":"2026-06-29T21:56:12.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556vm-test-run-nixbot-gitlab> gitlab # [ 161.952718] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cca94775852f0fa4cade874d","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cca94775852f0fa4cade874d: start","job_status":"start","queue_duration_s":0.750082,"scheduling_latency_s":0.750103}1557vm-test-run-nixbot-gitlab> gitlab # [ 161.954644] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cca94775852f0fa4cade874d","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cca94775852f0fa4cade874d: done: 0.015353 sec","job_status":"done","queue_duration_s":0.750082,"scheduling_latency_s":0.750103,"redis_calls":13,"redis_duration_s":0.003088,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015353,"completed_at":"2026-06-29T21:56:12.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558vm-test-run-nixbot-gitlab> gitlab # [ 161.973621] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e311a913ae67936fffe73f6b","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e311a913ae67936fffe73f6b: start","job_status":"start","queue_duration_s":0.766728,"scheduling_latency_s":0.76675}1559vm-test-run-nixbot-gitlab> gitlab # [ 161.993837] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e311a913ae67936fffe73f6b","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e311a913ae67936fffe73f6b: done: 0.015733 sec","job_status":"done","queue_duration_s":0.766728,"scheduling_latency_s":0.76675,"redis_calls":13,"redis_duration_s":0.002996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015733,"completed_at":"2026-06-29T21:56:12.501Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560vm-test-run-nixbot-gitlab> gitlab # [ 162.109543] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7056df13ab768c783406dec8","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7056df13ab768c783406dec8: start","job_status":"start","queue_duration_s":0.783783,"scheduling_latency_s":0.783804}1561vm-test-run-nixbot-gitlab> gitlab # [ 162.111775] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7056df13ab768c783406dec8","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7056df13ab768c783406dec8: done: 0.015608 sec","job_status":"done","queue_duration_s":0.783783,"scheduling_latency_s":0.783804,"redis_calls":13,"redis_duration_s":0.003326,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002436,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015608,"completed_at":"2026-06-29T21:56:12.518Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562vm-test-run-nixbot-gitlab> gitlab # [ 162.141351] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46829d7381b8e8da465af9b7","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46829d7381b8e8da465af9b7: start","job_status":"start","queue_duration_s":0.800691,"scheduling_latency_s":0.800713}1563vm-test-run-nixbot-gitlab> gitlab # [ 162.161186] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46829d7381b8e8da465af9b7","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46829d7381b8e8da465af9b7: done: 0.015439 sec","job_status":"done","queue_duration_s":0.800691,"scheduling_latency_s":0.800713,"redis_calls":13,"redis_duration_s":0.0030619999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015439,"completed_at":"2026-06-29T21:56:12.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564vm-test-run-nixbot-gitlab> gitlab # [ 162.248861] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9bbb3cfca277bf0c4fb7d28","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9bbb3cfca277bf0c4fb7d28: start","job_status":"start","queue_duration_s":0.817531,"scheduling_latency_s":0.817553}1565vm-test-run-nixbot-gitlab> gitlab # [ 162.250341] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9bbb3cfca277bf0c4fb7d28","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9bbb3cfca277bf0c4fb7d28: done: 0.015254 sec","job_status":"done","queue_duration_s":0.817531,"scheduling_latency_s":0.817553,"redis_calls":13,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015254,"completed_at":"2026-06-29T21:56:12.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566vm-test-run-nixbot-gitlab> gitlab # [ 162.306266] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"576d961d11e6a5f99d8e5d5c","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-576d961d11e6a5f99d8e5d5c: start","job_status":"start","queue_duration_s":0.834451,"scheduling_latency_s":0.834471}1567vm-test-run-nixbot-gitlab> gitlab # [ 162.326382] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"576d961d11e6a5f99d8e5d5c","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-576d961d11e6a5f99d8e5d5c: done: 0.015425 sec","job_status":"done","queue_duration_s":0.834451,"scheduling_latency_s":0.834471,"redis_calls":13,"redis_duration_s":0.0031650000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015425,"completed_at":"2026-06-29T21:56:12.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568vm-test-run-nixbot-gitlab> gitlab # [ 162.402075] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c84435dde2cb9d7a4a437f68","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c84435dde2cb9d7a4a437f68: start","job_status":"start","queue_duration_s":0.851282,"scheduling_latency_s":0.851303}1569vm-test-run-nixbot-gitlab> gitlab # [ 162.407150] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c84435dde2cb9d7a4a437f68","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c84435dde2cb9d7a4a437f68: done: 0.274166 sec","job_status":"done","queue_duration_s":0.851282,"scheduling_latency_s":0.851303,"redis_calls":13,"redis_duration_s":0.003275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001214,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.274166,"completed_at":"2026-06-29T21:56:12.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570vm-test-run-nixbot-gitlab> gitlab # [ 162.470907] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18c2e1fafebd4589621230aa","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18c2e1fafebd4589621230aa: start","job_status":"start","queue_duration_s":1.127659,"scheduling_latency_s":1.127689}1571vm-test-run-nixbot-gitlab> gitlab # [ 162.546205] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18c2e1fafebd4589621230aa","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18c2e1fafebd4589621230aa: done: 0.019431 sec","job_status":"done","queue_duration_s":1.127659,"scheduling_latency_s":1.127689,"redis_calls":13,"redis_duration_s":0.005724,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003868,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019431,"completed_at":"2026-06-29T21:56:12.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1572vm-test-run-nixbot-gitlab> gitlab # [ 162.551925] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e4121a6217f514dc816def2","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e4121a6217f514dc816def2: start","job_status":"start","queue_duration_s":1.149385,"scheduling_latency_s":1.149407}1573vm-test-run-nixbot-gitlab> gitlab # [ 162.572046] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e4121a6217f514dc816def2","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e4121a6217f514dc816def2: done: 0.018136 sec","job_status":"done","queue_duration_s":1.149385,"scheduling_latency_s":1.149407,"redis_calls":13,"redis_duration_s":0.0056110000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018136,"completed_at":"2026-06-29T21:56:12.886Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1574vm-test-run-nixbot-gitlab> gitlab # [ 162.691186] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20851b3e3a20765546bbc1d0","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20851b3e3a20765546bbc1d0: start","job_status":"start","queue_duration_s":1.169437,"scheduling_latency_s":1.169458}1575vm-test-run-nixbot-gitlab> gitlab # [ 162.692285] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20851b3e3a20765546bbc1d0","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20851b3e3a20765546bbc1d0: done: 0.021514 sec","job_status":"done","queue_duration_s":1.169437,"scheduling_latency_s":1.169458,"redis_calls":13,"redis_duration_s":0.008815,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005127,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003688,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021514,"completed_at":"2026-06-29T21:56:12.910Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576vm-test-run-nixbot-gitlab> gitlab # [ 162.716462] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4f51e566b8d6df1a32b091f","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4f51e566b8d6df1a32b091f: start","job_status":"start","queue_duration_s":1.193585,"scheduling_latency_s":1.193608}1577vm-test-run-nixbot-gitlab> gitlab # [ 162.736527] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4f51e566b8d6df1a32b091f","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4f51e566b8d6df1a32b091f: done: 0.044473 sec","job_status":"done","queue_duration_s":1.193585,"scheduling_latency_s":1.193608,"redis_calls":13,"redis_duration_s":0.015333,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.010812,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.044473,"completed_at":"2026-06-29T21:56:12.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578vm-test-run-nixbot-gitlab> gitlab # [ 162.837079] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a375477d1ec0e47a69995708","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a375477d1ec0e47a69995708: start","job_status":"start","queue_duration_s":1.241851,"scheduling_latency_s":1.241899}1579vm-test-run-nixbot-gitlab> gitlab # [ 162.838540] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a375477d1ec0e47a69995708","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a375477d1ec0e47a69995708: done: 0.020985 sec","job_status":"done","queue_duration_s":1.241851,"scheduling_latency_s":1.241899,"redis_calls":13,"redis_duration_s":0.003149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002248,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020985,"completed_at":"2026-06-29T21:56:12.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580vm-test-run-nixbot-gitlab> gitlab # [ 162.881204] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6b6e7ecd1d5b32c1c228d7d","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6b6e7ecd1d5b32c1c228d7d: start","job_status":"start","queue_duration_s":1.264447,"scheduling_latency_s":1.264468}1581vm-test-run-nixbot-gitlab> gitlab # [ 162.901088] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:12.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6b6e7ecd1d5b32c1c228d7d","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6b6e7ecd1d5b32c1c228d7d: done: 0.015147 sec","job_status":"done","queue_duration_s":1.264447,"scheduling_latency_s":1.264468,"redis_calls":13,"redis_duration_s":0.003005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015147,"completed_at":"2026-06-29T21:56:12.998Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582vm-test-run-nixbot-gitlab> gitlab # [ 162.964811] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8c9f3c361a2e407f1fbeaef","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8c9f3c361a2e407f1fbeaef: start","job_status":"start","queue_duration_s":1.280892,"scheduling_latency_s":1.280913}1583vm-test-run-nixbot-gitlab> gitlab # [ 162.984950] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8c9f3c361a2e407f1fbeaef","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8c9f3c361a2e407f1fbeaef: done: 0.015372 sec","job_status":"done","queue_duration_s":1.280892,"scheduling_latency_s":1.280913,"redis_calls":13,"redis_duration_s":0.003173,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015372,"completed_at":"2026-06-29T21:56:13.015Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584vm-test-run-nixbot-gitlab> gitlab # [ 163.048373] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72708a37a91e7f1086ae0d07","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72708a37a91e7f1086ae0d07: start","job_status":"start","queue_duration_s":1.297581,"scheduling_latency_s":1.297603}1585vm-test-run-nixbot-gitlab> gitlab # [ 163.129242] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72708a37a91e7f1086ae0d07","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72708a37a91e7f1086ae0d07: done: 0.015103 sec","job_status":"done","queue_duration_s":1.297581,"scheduling_latency_s":1.297603,"redis_calls":13,"redis_duration_s":0.0029460000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015103,"completed_at":"2026-06-29T21:56:13.031Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586vm-test-run-nixbot-gitlab> gitlab # [ 163.131288] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19d8860df8abb3e87caa9bb2","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19d8860df8abb3e87caa9bb2: start","job_status":"start","queue_duration_s":1.313944,"scheduling_latency_s":1.313966}1587vm-test-run-nixbot-gitlab> gitlab # [ 163.150118] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19d8860df8abb3e87caa9bb2","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19d8860df8abb3e87caa9bb2: done: 0.01498 sec","job_status":"done","queue_duration_s":1.313944,"scheduling_latency_s":1.313966,"redis_calls":13,"redis_duration_s":0.002953,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01498,"completed_at":"2026-06-29T21:56:13.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588vm-test-run-nixbot-gitlab> gitlab # [ 163.213517] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81f3ae55334307bd98195beb","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81f3ae55334307bd98195beb: start","job_status":"start","queue_duration_s":1.330384,"scheduling_latency_s":1.330405}1589vm-test-run-nixbot-gitlab> gitlab # [ 163.233896] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81f3ae55334307bd98195beb","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81f3ae55334307bd98195beb: done: 0.015744 sec","job_status":"done","queue_duration_s":1.330384,"scheduling_latency_s":1.330405,"redis_calls":13,"redis_duration_s":0.00296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015744,"completed_at":"2026-06-29T21:56:13.065Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590vm-test-run-nixbot-gitlab> gitlab # [ 163.297837] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cabb10e9a8a5d816ab6a66a8","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cabb10e9a8a5d816ab6a66a8: start","job_status":"start","queue_duration_s":1.347431,"scheduling_latency_s":1.347452}1591vm-test-run-nixbot-gitlab> gitlab # [ 163.317474] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cabb10e9a8a5d816ab6a66a8","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cabb10e9a8a5d816ab6a66a8: done: 0.015343 sec","job_status":"done","queue_duration_s":1.347431,"scheduling_latency_s":1.347452,"redis_calls":13,"redis_duration_s":0.003009,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015343,"completed_at":"2026-06-29T21:56:13.081Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592vm-test-run-nixbot-gitlab> gitlab # [ 163.422351] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dbca684d8451ed2560b4afe","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dbca684d8451ed2560b4afe: start","job_status":"start","queue_duration_s":1.364197,"scheduling_latency_s":1.364217}1593vm-test-run-nixbot-gitlab> gitlab # [ 163.424115] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dbca684d8451ed2560b4afe","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dbca684d8451ed2560b4afe: done: 0.015322 sec","job_status":"done","queue_duration_s":1.364197,"scheduling_latency_s":1.364217,"redis_calls":13,"redis_duration_s":0.0032029999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002266,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015322,"completed_at":"2026-06-29T21:56:13.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594vm-test-run-nixbot-gitlab> gitlab # [ 163.462915] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00befd10c2eeb0ec6d7448ef","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00befd10c2eeb0ec6d7448ef: start","job_status":"start","queue_duration_s":1.380834,"scheduling_latency_s":1.380857}1595vm-test-run-nixbot-gitlab> gitlab # [ 163.482847] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00befd10c2eeb0ec6d7448ef","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00befd10c2eeb0ec6d7448ef: done: 0.015733 sec","job_status":"done","queue_duration_s":1.380834,"scheduling_latency_s":1.380857,"redis_calls":13,"redis_duration_s":0.00302,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015733,"completed_at":"2026-06-29T21:56:13.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596vm-test-run-nixbot-gitlab> gitlab # [ 163.567905] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"422fd88ebf88e7ad9a1e4c83","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-422fd88ebf88e7ad9a1e4c83: start","job_status":"start","queue_duration_s":1.397961,"scheduling_latency_s":1.397984}1597vm-test-run-nixbot-gitlab> gitlab # [ 163.569241] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"422fd88ebf88e7ad9a1e4c83","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-422fd88ebf88e7ad9a1e4c83: done: 0.01524 sec","job_status":"done","queue_duration_s":1.397961,"scheduling_latency_s":1.397984,"redis_calls":13,"redis_duration_s":0.003124,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01524,"completed_at":"2026-06-29T21:56:13.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598vm-test-run-nixbot-gitlab> gitlab # [ 163.627346] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36d35aa36f627b8ca68d6e90","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36d35aa36f627b8ca68d6e90: start","job_status":"start","queue_duration_s":1.414489,"scheduling_latency_s":1.414508}1599vm-test-run-nixbot-gitlab> gitlab # [ 163.712677] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36d35aa36f627b8ca68d6e90","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36d35aa36f627b8ca68d6e90: done: 0.015203 sec","job_status":"done","queue_duration_s":1.414489,"scheduling_latency_s":1.414508,"redis_calls":13,"redis_duration_s":0.003142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015203,"completed_at":"2026-06-29T21:56:13.148Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600vm-test-run-nixbot-gitlab> gitlab # [ 163.714854] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63275aa0ac5d8ba91d182871","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63275aa0ac5d8ba91d182871: start","job_status":"start","queue_duration_s":1.431012,"scheduling_latency_s":1.431079}1601vm-test-run-nixbot-gitlab> gitlab # [ 163.728459] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63275aa0ac5d8ba91d182871","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63275aa0ac5d8ba91d182871: done: 0.0153 sec","job_status":"done","queue_duration_s":1.431012,"scheduling_latency_s":1.431079,"redis_calls":13,"redis_duration_s":0.00299,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0153,"completed_at":"2026-06-29T21:56:13.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602vm-test-run-nixbot-gitlab> gitlab # [ 163.858123] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c3ac42203f139f336fec70b","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c3ac42203f139f336fec70b: start","job_status":"start","queue_duration_s":1.447617,"scheduling_latency_s":1.447638}1603vm-test-run-nixbot-gitlab> gitlab # [ 163.859587] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c3ac42203f139f336fec70b","created_at":"2026-06-29T21:56:11.651Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-7aade0feca014286","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8d465c9ca163039d57a48e8ff69a8f00","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.719Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c3ac42203f139f336fec70b: done: 0.019716 sec","job_status":"done","queue_duration_s":1.447617,"scheduling_latency_s":1.447638,"redis_calls":13,"redis_duration_s":0.003518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001252,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002266,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019716,"completed_at":"2026-06-29T21:56:13.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1604vm-test-run-nixbot-gitlab> gitlab # [ 163.873383] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.188Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"92936bcadac51a5eb1aabe32","created_at":"2026-06-29T21:56:11.830Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-d0bba3f9300547c7","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.840Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-92936bcadac51a5eb1aabe32: start","job_status":"start","queue_duration_s":1.348566,"scheduling_latency_s":1.348597}1605vm-test-run-nixbot-gitlab> gitlab # [ 163.893895] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.204Z","message":"No existing merge request to be cleaned up."}1606vm-test-run-nixbot-gitlab> gitlab # [ 163.896278] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.214Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"92936bcadac51a5eb1aabe32","created_at":"2026-06-29T21:56:11.830Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-d0bba3f9300547c7","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.840Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-92936bcadac51a5eb1aabe32: done: 0.025348 sec","job_status":"done","queue_duration_s":1.348566,"scheduling_latency_s":1.348597,"redis_calls":13,"redis_duration_s":0.00322,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001252,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001444,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000412064,"duration_s":0.025348,"completed_at":"2026-06-29T21:56:13.214Z","load_balancing_strategy":"primary","db_duration_s":0.002945,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1607vm-test-run-nixbot-gitlab> gitlab # [ 164.003300] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.215Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6f94aaefe74d1dc4680c25cd","created_at":"2026-06-29T21:56:11.830Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-d0bba3f9300547c7","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.840Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6f94aaefe74d1dc4680c25cd: start","job_status":"start","queue_duration_s":1.375589,"scheduling_latency_s":1.375609}1608vm-test-run-nixbot-gitlab> gitlab # [ 164.004765] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.225Z","message":"No existing merge request to be cleaned up."}1609vm-test-run-nixbot-gitlab> gitlab # [ 164.006092] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.231Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6f94aaefe74d1dc4680c25cd","created_at":"2026-06-29T21:56:11.830Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-d0bba3f9300547c7","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.840Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6f94aaefe74d1dc4680c25cd: done: 0.015671 sec","job_status":"done","queue_duration_s":1.375589,"scheduling_latency_s":1.375609,"redis_calls":10,"redis_duration_s":0.0025529999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001553,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015671,"completed_at":"2026-06-29T21:56:13.231Z","load_balancing_strategy":"primary","db_duration_s":0.001393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610vm-test-run-nixbot-gitlab> gitlab # [ 164.044276] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.232Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"058268e06d39c25e323a1fcd","created_at":"2026-06-29T21:56:11.830Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-d0bba3f9300547c7","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.840Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-058268e06d39c25e323a1fcd: start","job_status":"start","queue_duration_s":1.392657,"scheduling_latency_s":1.392677}1611vm-test-run-nixbot-gitlab> gitlab # [ 164.064088] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.242Z","message":"No existing merge request to be cleaned up."}1612vm-test-run-nixbot-gitlab> gitlab # [ 164.066992] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.248Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"058268e06d39c25e323a1fcd","created_at":"2026-06-29T21:56:11.830Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-d0bba3f9300547c7","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.840Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-058268e06d39c25e323a1fcd: done: 0.015635 sec","job_status":"done","queue_duration_s":1.392657,"scheduling_latency_s":1.392677,"redis_calls":10,"redis_duration_s":0.00235,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015635,"completed_at":"2026-06-29T21:56:13.248Z","load_balancing_strategy":"primary","db_duration_s":0.001455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1613vm-test-run-nixbot-gitlab> gitlab # [ 164.129894] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.249Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"12b059c91fcc5a7f61bdcaf6","created_at":"2026-06-29T21:56:11.830Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-d0bba3f9300547c7","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.840Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-12b059c91fcc5a7f61bdcaf6: start","job_status":"start","queue_duration_s":1.409704,"scheduling_latency_s":1.409726}1614vm-test-run-nixbot-gitlab> gitlab # [ 164.212333] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.259Z","message":"No existing merge request to be cleaned up."}1615vm-test-run-nixbot-gitlab> gitlab # [ 164.213473] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:13.265Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"12b059c91fcc5a7f61bdcaf6","created_at":"2026-06-29T21:56:11.830Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-d0bba3f9300547c7","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"94866d1f7f25fe8dbcdb118f656d1648","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:11.840Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-12b059c91fcc5a7f61bdcaf6: done: 0.015439 sec","job_status":"done","queue_duration_s":1.409704,"scheduling_latency_s":1.409726,"redis_calls":10,"redis_duration_s":0.002484,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001491,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015439,"completed_at":"2026-06-29T21:56:13.265Z","load_balancing_strategy":"primary","db_duration_s":0.001396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1616vm-test-run-nixbot-gitlab> gitlab # [ 164.542594] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 6022016474 wd_nsec: 60220138891617vm-test-run-nixbot-gitlab> gitlab # [ 177.415207] setup-gitlab-start[2781]: W, [2026-06-29T21:56:30.407139 #2781] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1618vm-test-run-nixbot-gitlab> gitlab # [ 200.272541] runuser[2780]: pam_unix(runuser:session): session closed for user gitlab1619vm-test-run-nixbot-gitlab> gitlab # [ 200.279806] setup-gitlab-start[2992]: ++ jq -r .token1620vm-test-run-nixbot-gitlab> gitlab # [ 200.283255] setup-gitlab-start[2993]: +++ date -d '+7 days' +%F1621vm-test-run-nixbot-gitlab> gitlab # [ 200.286664] setup-gitlab-start[2991]: ++ curl -fs -X POST -H 'Authorization: Bearer 41e5b04f9a22f1e7a2297c658842ad5c3d5a9a223cbce6be5ef13fa861567af6' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-06"}' http://gitlab/api/v4/users/1/personal_access_tokens1622vm-test-run-nixbot-gitlab> gitlab # [ 200.740768] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:53.732Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"50e0f3531998ee0dd815fdd4","created_at":"2026-06-29T21:56:53.675Z","trace_propagation_headers":{"sentry-trace":"f20e15b3f00d4de89c812f378684ce0f-fd7303d43ba44004","baggage":"sentry-trace_id=f20e15b3f00d4de89c812f378684ce0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58DECNK87Q48HZRHSSBW","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:34353274e7fb3dcc67978427edde804cdf16fcc6199a4ee97c68f52208252e31","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:53.726Z","job_size_bytes":451,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-50e0f3531998ee0dd815fdd4: start","job_status":"start","queue_duration_s":0.005101,"scheduling_latency_s":0.005175}1623vm-test-run-nixbot-gitlab> gitlab # [ 200.775539] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:56:53 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 4721624vm-test-run-nixbot-gitlab> gitlab # [ 200.780959] setup-gitlab-start[2738]: + TOKEN=glpat-xmEyD4-0E_mTeNU35EiwQW86MQp1OjEH.01.0w0zt1x6n1625vm-test-run-nixbot-gitlab> gitlab # [ 200.783212] setup-gitlab-start[2738]: + test -n glpat-xmEyD4-0E_mTeNU35EiwQW86MQp1OjEH.01.0w0zt1x6n1626vm-test-run-nixbot-gitlab> gitlab # [ 200.785303] setup-gitlab-start[2738]: + test glpat-xmEyD4-0E_mTeNU35EiwQW86MQp1OjEH.01.0w0zt1x6n '!=' null1627vm-test-run-nixbot-gitlab> gitlab # [ 200.787545] setup-gitlab-start[2738]: + mkdir -p /var/lib/secrets1628vm-test-run-nixbot-gitlab> gitlab # [ 200.789121] setup-gitlab-start[2738]: + echo glpat-xmEyD4-0E_mTeNU35EiwQW86MQp1OjEH.01.0w0zt1x6n1629vm-test-run-nixbot-gitlab> gitlab # [ 200.791196] setup-gitlab-start[2738]: + chmod 644 /var/lib/secrets/gitlab-token1630vm-test-run-nixbot-gitlab> gitlab # [ 200.793057] setup-gitlab-start[2738]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-xmEyD4-0E_mTeNU35EiwQW86MQp1OjEH.01.0w0zt1x6n' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1631vm-test-run-nixbot-gitlab> gitlab # [ 200.942619] sidekiq-cluster[2173]: {"severity":"WARN","time":"2026-06-29T21:56:53.933Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"50e0f3531998ee0dd815fdd4","created_at":"2026-06-29T21:56:53.675Z","trace_propagation_headers":{"sentry-trace":"f20e15b3f00d4de89c812f378684ce0f-fd7303d43ba44004","baggage":"sentry-trace_id=f20e15b3f00d4de89c812f378684ce0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58DECNK87Q48HZRHSSBW","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:34353274e7fb3dcc67978427edde804cdf16fcc6199a4ee97c68f52208252e31","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:53.726Z","job_size_bytes":451,"pid":2173,"sidekiq_tid":"lyt5","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-50e0f3531998ee0dd815fdd4: fail: 0.199367 sec","job_status":"fail","queue_duration_s":0.005101,"scheduling_latency_s":0.005175,"redis_calls":11,"redis_duration_s":0.00724,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001965,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003852,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.119184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001141765,"duration_s":0.199367,"completed_at":"2026-06-29T21:56:53.931Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.048331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1632vm-test-run-nixbot-gitlab> gitlab # [ 201.539603] gitaly[1387]: time="2026-06-29T21:56:54.437Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.436" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1633vm-test-run-nixbot-gitlab> gitlab # [ 201.541932] gitaly[1387]: time="2026-06-29T21:56:54.437Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.436" grpc.time_ms=0.427 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1634vm-test-run-nixbot-gitlab> gitlab # [ 201.543212] gitaly[1387]: time="2026-06-29T21:56:54.514Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.083" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.514" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1635vm-test-run-nixbot-gitlab> gitlab # [ 201.545800] gitaly[1387]: time="2026-06-29T21:56:54.514Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.083" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.514" grpc.time_ms=0.32 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1636vm-test-run-nixbot-gitlab> gitlab # [ 201.616314] gitaly[1387]: time="2026-06-29T21:56:54.555Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X diskcache=a8af7675-9521-41ca-b09e-e759721b5c29 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T21:57:04.091" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.522" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1637vm-test-run-nixbot-gitlab> gitlab # [ 201.617537] gitaly[1387]: time="2026-06-29T21:56:54.556Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=32 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T21:57:04.091" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.522" grpc.time_ms=33.857 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1638vm-test-run-nixbot-gitlab> gitlab # [ 201.618493] gitaly[1387]: time="2026-06-29T21:56:54.566Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.135" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.565" pid=1387 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1639vm-test-run-nixbot-gitlab> gitlab # [ 201.684438] gitaly[1387]: time="2026-06-29T21:56:54.677Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.135" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.565" grpc.time_ms=111.503 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1640vm-test-run-nixbot-gitlab> gitlab # [ 201.697588] gitaly[1387]: time="2026-06-29T21:56:54.690Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:57:04.251" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.682" grpc.time_ms=7.848 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1641vm-test-run-nixbot-gitlab> gitlab # [ 201.734655] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:54.727Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5c45f3bbc7c99fd307b3f3da","created_at":"2026-06-29T21:56:54.699Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:54.725Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5c45f3bbc7c99fd307b3f3da: start","job_status":"start","queue_duration_s":0.002333,"scheduling_latency_s":0.002376}1642vm-test-run-nixbot-gitlab> gitlab # [ 201.747354] gitaly[1387]: time="2026-06-29T21:56:54.740Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.740" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1643vm-test-run-nixbot-gitlab> gitlab # [ 201.749244] gitaly[1387]: time="2026-06-29T21:56:54.740Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.740" grpc.time_ms=0.472 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1644vm-test-run-nixbot-gitlab> gitlab # [ 201.782141] gitaly[1387]: time="2026-06-29T21:56:54.774Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X diskcache=b7adad9f-da3f-497c-8031-40dce74ae661 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T21:57:04.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.743" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1645vm-test-run-nixbot-gitlab> gitlab # [ 201.783535] gitaly[1387]: time="2026-06-29T21:56:54.775Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=323 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T21:57:04.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.743" grpc.time_ms=31.69 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1646vm-test-run-nixbot-gitlab> gitlab # [ 201.791135] gitaly[1387]: time="2026-06-29T21:56:54.784Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.053" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.783" pid=1387 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1647vm-test-run-nixbot-gitlab> gitlab # [ 201.791913] gitaly[1387]: time="2026-06-29T21:56:54.784Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.053" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.783" grpc.time_ms=0.428 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1648vm-test-run-nixbot-gitlab> gitlab # [ 201.797835] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:54.786Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5c45f3bbc7c99fd307b3f3da","created_at":"2026-06-29T21:56:54.699Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:54.725Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5c45f3bbc7c99fd307b3f3da: done: 0.059091 sec","job_status":"done","queue_duration_s":0.002333,"scheduling_latency_s":0.002376,"redis_calls":10,"redis_duration_s":0.003404,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002072,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001332,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.035983065,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.059091,"completed_at":"2026-06-29T21:56:54.786Z","load_balancing_strategy":"primary","db_duration_s":0.010238,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1649vm-test-run-nixbot-gitlab> gitlab # [ 201.931083] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:54.788Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"3ac9a3af2ca83480a830951f","created_at":"2026-06-29T21:56:54.731Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:54.735Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-3ac9a3af2ca83480a830951f: start","job_status":"start","queue_duration_s":0.052203,"scheduling_latency_s":0.052231}1650vm-test-run-nixbot-gitlab> gitlab # [ 201.932397] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:54.868Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"3ac9a3af2ca83480a830951f","created_at":"2026-06-29T21:56:54.731Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:54.735Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-3ac9a3af2ca83480a830951f: done: 0.079942 sec","job_status":"done","queue_duration_s":0.052203,"scheduling_latency_s":0.052231,"redis_calls":14,"redis_duration_s":0.003594,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000762,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001654,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001178,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.057532452,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.079942,"completed_at":"2026-06-29T21:56:54.867Z","load_balancing_strategy":"primary","db_duration_s":0.016814,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1651vm-test-run-nixbot-gitlab> gitlab # [ 201.962137] gitaly[1387]: time="2026-06-29T21:56:54.910Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=117 command.spawn_token_fork_ms=112 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:57:04.061" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.791" grpc.time_ms=118.673 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1652vm-test-run-nixbot-gitlab> gitlab # [ 202.003896] gitaly[1387]: time="2026-06-29T21:56:54.996Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:04.266" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:54.996" grpc.time_ms=0.184 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1653vm-test-run-nixbot-gitlab> gitlab # [ 202.011394] gitaly[1387]: time="2026-06-29T21:56:55.004Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T21:57:05.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:55.004" grpc.time_ms=0.229 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1654vm-test-run-nixbot-gitlab> gitlab # [ 202.179109] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:55.171Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"703e956947163e3d98174839","created_at":"2026-06-29T21:56:55.162Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:55.169Z","job_size_bytes":5,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-703e956947163e3d98174839: start","job_status":"start","queue_duration_s":0.002076,"scheduling_latency_s":0.002116}1655vm-test-run-nixbot-gitlab> gitlab # [ 202.313217] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:55.305Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"703e956947163e3d98174839","created_at":"2026-06-29T21:56:55.162Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:55.169Z","job_size_bytes":5,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-703e956947163e3d98174839: done: 0.134437 sec","job_status":"done","queue_duration_s":0.002076,"scheduling_latency_s":0.002116,"redis_calls":17,"redis_duration_s":0.007137,"redis_read_bytes":144,"redis_write_bytes":3399,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.002216,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":884,"redis_queues_calls":2,"redis_queues_duration_s":0.000668,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1257,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003344,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":992,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":16,"db_write_count":1,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000748419,"duration_s":0.134437,"completed_at":"2026-06-29T21:56:55.305Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02883,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1656vm-test-run-nixbot-gitlab> gitlab # [ 202.420560] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:55.413Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"8461022fca1bebbffab65999","created_at":"2026-06-29T21:56:55.273Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:55.283Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-8461022fca1bebbffab65999: start","job_status":"start","queue_duration_s":0.129932,"scheduling_latency_s":0.129983}1657vm-test-run-nixbot-gitlab> gitlab # [ 202.438779] gitaly[1387]: time="2026-06-29T21:56:55.431Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP58XB4346YB4A5R75FV0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:57:05.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:55.415" grpc.time_ms=15.432 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1658vm-test-run-nixbot-gitlab> gitlab # [ 202.710563] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:55.702Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"8461022fca1bebbffab65999","created_at":"2026-06-29T21:56:55.273Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:55.283Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-8461022fca1bebbffab65999: done: 0.290035 sec","job_status":"done","queue_duration_s":0.129932,"scheduling_latency_s":0.129983,"redis_calls":11,"redis_duration_s":0.006894,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.002155,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003584,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001155,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.177,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.177,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.133901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000559009,"duration_s":0.290035,"completed_at":"2026-06-29T21:56:55.702Z","load_balancing_strategy":"primary","db_duration_s":0.099872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1659vm-test-run-nixbot-gitlab> gitlab # [ 202.820194] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:55.811Z","retry":25,"queue":"default","version":0,"store":null,"class":"Ci::PipelineSchedules::DeactivateSchedulesWorker","args":["ProjectAuthorizations::AuthorizationsAddedEvent","[{\"project_ids\"=\u003e[1], \"user_ids\"=\u003e[1]}]"],"jid":"88b3cd02c20961326c666ae8","created_at":"2026-06-29T21:56:55.282Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-4b45d74cd9a94c0a","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39ab203788ffde42edba0d838663a2477aee99a8f3196d11bf650021a1ee799d","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:55.294Z","job_size_bytes":88,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineSchedules::DeactivateSchedulesWorker JID-88b3cd02c20961326c666ae8: start","job_status":"start","queue_duration_s":0.516649,"scheduling_latency_s":0.516719}1660vm-test-run-nixbot-gitlab> gitlab # [ 202.882748] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:55.874Z","retry":25,"queue":"default","version":0,"store":null,"class":"Ci::PipelineSchedules::DeactivateSchedulesWorker","args":["ProjectAuthorizations::AuthorizationsAddedEvent","[{\"project_ids\"=\u003e[1], \"user_ids\"=\u003e[1]}]"],"jid":"88b3cd02c20961326c666ae8","created_at":"2026-06-29T21:56:55.282Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-4b45d74cd9a94c0a","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39ab203788ffde42edba0d838663a2477aee99a8f3196d11bf650021a1ee799d","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:55.294Z","job_size_bytes":88,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineSchedules::DeactivateSchedulesWorker JID-88b3cd02c20961326c666ae8: done: 0.063677 sec","job_status":"done","queue_duration_s":0.516649,"scheduling_latency_s":0.516719,"redis_calls":10,"redis_duration_s":0.006681,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004177,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000963251,"duration_s":0.063677,"completed_at":"2026-06-29T21:56:55.874Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008234,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1661vm-test-run-nixbot-gitlab> gitlab # [ 202.963687] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:55.955Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"f929927efee5c37fa7ac7ee7","created_at":"2026-06-29T21:56:55.289Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:55.296Z","job_size_bytes":89,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-f929927efee5c37fa7ac7ee7: start","job_status":"start","queue_duration_s":0.659833,"scheduling_latency_s":0.659884}1662vm-test-run-nixbot-gitlab> gitlab # [ 203.233411] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:56.225Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"f929927efee5c37fa7ac7ee7","created_at":"2026-06-29T21:56:55.289Z","trace_propagation_headers":{"sentry-trace":"b786ad1901174dddbaa02d504fe81503-5758bd0ba10645d5","baggage":"sentry-trace_id=b786ad1901174dddbaa02d504fe81503,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58XB4346YB4A5R75FV0X","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:55.296Z","job_size_bytes":89,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-f929927efee5c37fa7ac7ee7: done: 0.269464 sec","job_status":"done","queue_duration_s":0.659833,"scheduling_latency_s":0.659884,"redis_calls":11,"redis_duration_s":0.007182,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001916,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003869,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001397,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000892292,"duration_s":0.269464,"completed_at":"2026-06-29T21:56:56.225Z","load_balancing_strategy":"primary","db_duration_s":0.034273,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1663vm-test-run-nixbot-gitlab> gitlab # [ 203.860321] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:56:56 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 30471664vm-test-run-nixbot-gitlab> gitlab # [ 203.865171] setup-gitlab-start[2738]: + mkdir -p /tmp/test-flake1665vm-test-run-nixbot-gitlab> gitlab # [ 203.867398] setup-gitlab-start[2738]: + cd /tmp/test-flake1666vm-test-run-nixbot-gitlab> gitlab # [ 203.870335] setup-gitlab-start[2738]: + git init -b master1667vm-test-run-nixbot-gitlab> gitlab # [ 203.885994] setup-gitlab-start[3045]: Initialized empty Git repository in /tmp/test-flake/.git/1668vm-test-run-nixbot-gitlab> gitlab # [ 203.889417] setup-gitlab-start[2738]: + git config user.name test1669vm-test-run-nixbot-gitlab> gitlab # [ 203.899398] setup-gitlab-start[2738]: + git config user.email test@example.com1670vm-test-run-nixbot-gitlab> gitlab # [ 203.902206] setup-gitlab-start[2738]: + cat1671vm-test-run-nixbot-gitlab> gitlab # [ 203.904853] setup-gitlab-start[2738]: + git add flake.nix1672vm-test-run-nixbot-gitlab> gitlab # [ 203.907887] setup-gitlab-start[2738]: + git commit -m 'initial commit'1673vm-test-run-nixbot-gitlab> gitlab # [ 203.916851] setup-gitlab-start[3050]: [master (root-commit) ae94137] initial commit1674vm-test-run-nixbot-gitlab> gitlab # [ 203.918177] setup-gitlab-start[3050]: 1 file changed, 10 insertions(+)1675vm-test-run-nixbot-gitlab> gitlab # [ 203.920829] setup-gitlab-start[3050]: create mode 100644 flake.nix1676vm-test-run-nixbot-gitlab> gitlab # [ 203.926581] setup-gitlab-start[2738]: + git remote add origin http://oauth2:glpat-xmEyD4-0E_mTeNU35EiwQW86MQp1OjEH.01.0w0zt1x6n@gitlab/root/test-flake.git1677vm-test-run-nixbot-gitlab> gitlab # [ 203.941279] setup-gitlab-start[2738]: + git push -u origin master1678vm-test-run-nixbot-gitlab> gitlab # [ 204.075066] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:56:57 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 951679vm-test-run-nixbot-gitlab> gitlab # [ 204.349167] gitaly[1387]: time="2026-06-29T21:56:57.341Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5C3NHCYBXG8THKXTGRRJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:56:57.325" grpc.time_ms=15.351 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1680vm-test-run-nixbot-gitlab> gitlab # [ 204.360377] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:56:57 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 2741681vm-test-run-nixbot-gitlab> gitlab # [ 204.731992] gitaly[1387]: time="2026-06-29T21:56:57.724Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:57:07.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:57.713" grpc.time_ms=10.638 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1682vm-test-run-nixbot-gitlab> gitlab # [ 204.773982] gitaly[1387]: time="2026-06-29T21:56:57.767Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T21:57:27.054" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T21:56:57.754" grpc.time_ms=11.994 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1683vm-test-run-nixbot-gitlab> gitlab # [ 204.792398] gitaly[1387]: time="2026-06-29T21:56:57.785Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWAP5CDSV6D4Y3E843986P0M duration_ms=159ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/allowed"1684vm-test-run-nixbot-gitlab> gitlab # [ 204.816081] gitaly[1387]: time="2026-06-29T21:56:57.808Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWAP5CDSV6D4Y3E843986P0M duration_ms=21ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/pre_receive"1685vm-test-run-nixbot-gitlab> gitlab # [ 204.819097] gitaly[1387]: time="2026-06-29T21:56:57.811Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5321 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:56:57.625" grpc.time_ms=185.677 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1686vm-test-run-nixbot-gitlab> gitlab # [ 204.862829] gitaly[1387]: time="2026-06-29T21:56:57.855Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:56:57.855" grpc.time_ms=0.631 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1687vm-test-run-nixbot-gitlab> gitlab # [ 204.905483] gitaly[1387]: time="2026-06-29T21:56:57.898Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:56:57.897" grpc.time_ms=0.504 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1688vm-test-run-nixbot-gitlab> gitlab # [ 204.945077] gitaly[1387]: time="2026-06-29T21:56:57.938Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:56:57.937" grpc.time_ms=0.452 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1689vm-test-run-nixbot-gitlab> gitlab # [ 205.028153] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:58.018Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 ae941377e143a952c786905177b2065f7acae3f5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"d58172cbd850b7b63654238c","created_at":"2026-06-29T21:56:58.011Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-30a231c2c4004371","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.015Z","job_size_bytes":185,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-d58172cbd850b7b63654238c: start","job_status":"start","queue_duration_s":0.002409,"scheduling_latency_s":0.002478}1690vm-test-run-nixbot-gitlab> gitlab # [ 205.063446] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:58.050Z","message":"changes: \"0000000000000000000000000000000000000000 ae941377e143a952c786905177b2065f7acae3f5 refs/heads/master\\n\""}1691vm-test-run-nixbot-gitlab> gitlab # [ 205.067760] gitaly[1387]: time="2026-06-29T21:56:58.059Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=18 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:57:08.037" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:58.037" grpc.time_ms=21.33 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1692vm-test-run-nixbot-gitlab> gitlab # [ 205.090500] gitaly[1387]: time="2026-06-29T21:56:58.083Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1565 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T21:57:08.065" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:56:58.066" grpc.time_ms=17.113 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1693vm-test-run-nixbot-gitlab> gitlab # [ 205.114873] gitaly[1387]: time="2026-06-29T21:56:58.096Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:57:08.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:58.089" grpc.time_ms=7.567 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1694vm-test-run-nixbot-gitlab> gitlab # [ 205.137830] gitaly[1387]: time="2026-06-29T21:56:58.098Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWAP5CDSV6D4Y3E843986P0M duration_ms=118ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/post_receive"1695vm-test-run-nixbot-gitlab> gitlab # [ 205.138912] gitaly[1387]: time="2026-06-29T21:56:58.099Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4815 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:56:57.979" grpc.time_ms=119.791 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1696vm-test-run-nixbot-gitlab> gitlab # [ 205.140060] gitaly[1387]: time="2026-06-29T21:56:58.117Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M diskcache=1e199d2e-1b39-44c4-9899-b767c6b464da grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:56:57.511" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1697vm-test-run-nixbot-gitlab> gitlab # [ 205.141110] setup-gitlab-start[3054]: To http://gitlab/root/test-flake.git1698vm-test-run-nixbot-gitlab> gitlab # [ 205.142067] setup-gitlab-start[3054]: * [new branch] master -> master1699vm-test-run-nixbot-gitlab> gitlab # [ 205.142936] setup-gitlab-start[3054]: branch 'master' set up to track 'origin/master'.1700vm-test-run-nixbot-gitlab> gitlab # [ 205.145542] gitlab-workhorse[1305]: time="2026-06-29T21:56:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521701vm-test-run-nixbot-gitlab> gitlab # [ 205.147124] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:56:58 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 7161702vm-test-run-nixbot-gitlab> gitlab # [ 205.148837] gitaly[1387]: time="2026-06-29T21:56:58.117Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=211 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13077 command.oublock=32 command.real_time_ms=592 command.spawn_token_fork_ms=4 command.system_time_ms=152 command.user_time_ms=59 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:56:57.511" grpc.time_ms=605.783 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1703vm-test-run-nixbot-gitlab> gitlab # [ 205.149984] gitaly[1387]: time="2026-06-29T21:56:58.143Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:08.055" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:56:58.135" grpc.time_ms=7.12 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1704vm-test-run-nixbot-gitlab> gitlab # [ 205.184812] gitaly[1387]: time="2026-06-29T21:56:58.177Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T21:57:08.089" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:56:58.169" grpc.time_ms=8.101 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1705vm-test-run-nixbot-gitlab> gitlab # [ 205.218629] gitaly[1387]: time="2026-06-29T21:56:58.211Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3941 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:56:58.210" grpc.time_ms=1.282 pid=1387 span.kind=server system=grpc1706vm-test-run-nixbot-gitlab> gitlab # [ 205.244349] gitaly[1387]: time="2026-06-29T21:56:58.237Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3943 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:56:58.236" grpc.time_ms=0.424 pid=1387 span.kind=server system=grpc1707vm-test-run-nixbot-gitlab> gitlab # [ 205.260211] gitaly[1387]: time="2026-06-29T21:56:58.253Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M diskcache=cc93d69d-0ebc-4a37-8c08-cb0b9b0b4698 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T21:57:08.105" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:58.185" pid=1387 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1708vm-test-run-nixbot-gitlab> gitlab # [ 205.261416] gitaly[1387]: time="2026-06-29T21:56:58.253Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=59 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5058 command.oublock=8 command.real_time_ms=57 command.spawn_token_fork_ms=0 command.system_time_ms=45 command.user_time_ms=14 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T21:57:08.105" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:58.185" grpc.time_ms=67.411 pid=1387 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1709vm-test-run-nixbot-gitlab> gitlab # [ 205.277516] gitaly[1387]: time="2026-06-29T21:56:58.270Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T21:57:08.189" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:58.269" grpc.time_ms=0.177 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1710vm-test-run-nixbot-gitlab> gitlab # [ 205.292109] gitaly[1387]: time="2026-06-29T21:56:58.285Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T21:57:08.195" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:56:58.275" grpc.time_ms=9.336 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1711vm-test-run-nixbot-gitlab> gitlab # [ 205.349382] gitaly[1387]: time="2026-06-29T21:56:58.341Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-29T21:57:28.036" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:58.336" grpc.time_ms=5.154 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1712vm-test-run-nixbot-gitlab> gitlab # [ 205.368719] gitaly[1387]: time="2026-06-29T21:56:58.361Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T21:57:08.280" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:58.361" grpc.time_ms=0.272 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1713vm-test-run-nixbot-gitlab> gitlab # [ 205.375238] gitaly[1387]: time="2026-06-29T21:56:58.368Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T21:57:08.282" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T21:56:58.363" grpc.time_ms=4.82 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1714vm-test-run-nixbot-gitlab> gitlab # [ 205.392674] gitaly[1387]: time="2026-06-29T21:56:58.385Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:08.299" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:56:58.379" grpc.time_ms=5.97 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1715vm-test-run-nixbot-gitlab> gitlab # [ 205.402591] gitaly[1387]: time="2026-06-29T21:56:58.395Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:08.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:56:58.389" grpc.time_ms=6.046 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1716vm-test-run-nixbot-gitlab> gitlab # [ 205.422138] gitaly[1387]: time="2026-06-29T21:56:58.415Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T21:57:28.104" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:58.404" grpc.time_ms=10.083 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1717vm-test-run-nixbot-gitlab> gitlab # [ 205.487735] systemd[1]: Finished setup-gitlab.service.1718vm-test-run-nixbot-gitlab> gitlab # [ 205.491824] systemd[1]: Starting nixbot CI service...1719vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 206.57 seconds)1720vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1721vm-test-run-nixbot-gitlab> gitlab # [ 205.691481] gitaly[1387]: time="2026-06-29T21:56:58.684Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T21:57:08.001" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:58.681" grpc.time_ms=2.445 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1722vm-test-run-nixbot-gitlab> gitlab # [ 205.724892] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:58.717Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 ae941377e143a952c786905177b2065f7acae3f5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"d58172cbd850b7b63654238c","created_at":"2026-06-29T21:56:58.011Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-30a231c2c4004371","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.015Z","job_size_bytes":185,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-d58172cbd850b7b63654238c: done: 0.699644 sec","job_status":"done","queue_duration_s":0.002409,"scheduling_latency_s":0.002478,"gitaly_calls":13,"gitaly_duration_s":0.208219,"redis_calls":207,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.057202,"redis_read_bytes":2091,"redis_write_bytes":20170,"redis_cache_calls":14,"redis_cache_duration_s":0.00503,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":39,"redis_feature_flag_duration_s":0.012911,"redis_feature_flag_read_bytes":1690,"redis_feature_flag_write_bytes":4640,"redis_queues_calls":8,"redis_queues_duration_s":0.001611,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4933,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002518,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.021953,"redis_repository_cache_read_bytes":233,"redis_repository_cache_write_bytes":6319,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.013179,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":50,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":50,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":50,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.195,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.21,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.410108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000781664,"exclusive_lock_hold_duration_s":0.029015699,"duration_s":0.699644,"completed_at":"2026-06-29T21:56:58.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057639,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1723vm-test-run-nixbot-gitlab> gitlab # [ 205.816157] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:58.722Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"3eaac616141ec7790126031c","created_at":"2026-06-29T21:56:58.147Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-a432a0aeb1ea4a40","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.151Z","job_size_bytes":46,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3eaac616141ec7790126031c: start","job_status":"start","queue_duration_s":0.570917,"scheduling_latency_s":0.570941}1724vm-test-run-nixbot-gitlab> gitlab # [ 205.817440] gitaly[1387]: time="2026-06-29T21:56:58.766Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-30T03:56:58.763" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:56:58.764" grpc.time_ms=2.447 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1725vm-test-run-nixbot-gitlab> gitlab # [ 205.894095] gitaly[1387]: time="2026-06-29T21:56:58.886Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T21:57:28.578" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:58.878" grpc.time_ms=7.808 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1726vm-test-run-nixbot-gitlab> gitlab # [ 205.916613] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:58.909Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"3eaac616141ec7790126031c","created_at":"2026-06-29T21:56:58.147Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-a432a0aeb1ea4a40","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.151Z","job_size_bytes":46,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3eaac616141ec7790126031c: done: 0.186569 sec","job_status":"done","queue_duration_s":0.570917,"scheduling_latency_s":0.570941,"gitaly_calls":2,"gitaly_duration_s":0.119185,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010296,"redis_read_bytes":68,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002312,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000571,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00244,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003837,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001136,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000344178,"duration_s":0.186569,"completed_at":"2026-06-29T21:56:58.909Z","load_balancing_strategy":"primary","db_duration_s":0.011001,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1727vm-test-run-nixbot-gitlab> gitlab # [ 205.922506] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:58.910Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"088a280b6d6dccd3ac1c68e4","created_at":"2026-06-29T21:56:58.162Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-a432a0aeb1ea4a40","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.165Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-088a280b6d6dccd3ac1c68e4: start","job_status":"start","queue_duration_s":0.744936,"scheduling_latency_s":0.74496}1728vm-test-run-nixbot-gitlab> gitlab # [ 205.964638] gitaly[1387]: time="2026-06-29T21:56:58.957Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M diskcache=48561037-490a-4edd-a490-bcc3b153e88b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T03:56:58.931" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:58.931" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1729vm-test-run-nixbot-gitlab> gitlab # [ 205.966559] gitaly[1387]: time="2026-06-29T21:56:58.957Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1280 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T03:56:58.931" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:58.931" grpc.time_ms=25.908 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1730vm-test-run-nixbot-gitlab> gitlab # [ 206.090694] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:59.082Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"088a280b6d6dccd3ac1c68e4","created_at":"2026-06-29T21:56:58.162Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-a432a0aeb1ea4a40","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.165Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-088a280b6d6dccd3ac1c68e4: done: 0.172074 sec","job_status":"done","queue_duration_s":0.744936,"scheduling_latency_s":0.74496,"gitaly_calls":1,"gitaly_duration_s":0.124463,"redis_calls":15,"redis_duration_s":0.0037210000000000003,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000622,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001222,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.151188769,"duration_s":0.172074,"completed_at":"2026-06-29T21:56:59.082Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012561,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1731vm-test-run-nixbot-gitlab> gitlab # [ 206.174475] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:59.166Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"c5d867685041cc048c64774c","created_at":"2026-06-29T21:56:58.265Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-a432a0aeb1ea4a40","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.266Z","job_size_bytes":23,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c5d867685041cc048c64774c: start","job_status":"start","queue_duration_s":0.89978,"scheduling_latency_s":0.899849}1732vm-test-run-nixbot-gitlab> gitlab # [ 206.259698] gitaly[1387]: time="2026-06-29T21:56:59.252Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T21:57:29.042" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:59.242" grpc.time_ms=9.545 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1733vm-test-run-nixbot-gitlab> gitlab # [ 206.269873] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:59.262Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"c5d867685041cc048c64774c","created_at":"2026-06-29T21:56:58.265Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-a432a0aeb1ea4a40","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.266Z","job_size_bytes":23,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c5d867685041cc048c64774c: done: 0.096105 sec","job_status":"done","queue_duration_s":0.89978,"scheduling_latency_s":0.899849,"gitaly_calls":1,"gitaly_duration_s":0.013883,"redis_calls":16,"redis_duration_s":0.0060680000000000005,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000296,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002739,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001196,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.096105,"completed_at":"2026-06-29T21:56:59.262Z","load_balancing_strategy":"primary","db_duration_s":0.003905,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1734vm-test-run-nixbot-gitlab> gitlab # [ 206.282305] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:56:59.264Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a9557bc2a58f222f7058fcfb","created_at":"2026-06-29T21:56:58.647Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-a432a0aeb1ea4a40","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:66c681c6a4a2153af802b5ff028fba75d942bb28512196cdab47b736112f5f67","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.654Z","job_size_bytes":467,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a9557bc2a58f222f7058fcfb: start","job_status":"start","queue_duration_s":0.60935,"scheduling_latency_s":0.609378}1735vm-test-run-nixbot-gitlab> gitlab # [ 206.310258] gitaly[1387]: time="2026-06-29T21:56:59.298Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:29.097" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:59.297" grpc.time_ms=0.819 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1736vm-test-run-nixbot-gitlab> gitlab # [ 206.401692] gitaly[1387]: time="2026-06-29T21:56:59.394Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T21:57:09.000" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:56:59.301" grpc.time_ms=92.812 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1737vm-test-run-nixbot-gitlab> gitlab # [ 206.520992] gitaly[1387]: time="2026-06-29T21:56:59.513Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T21:57:29.310" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:59.511" grpc.time_ms=1.942 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1738vm-test-run-nixbot-gitlab> gitlab # [ 206.575684] gitaly[1387]: time="2026-06-29T21:56:59.568Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=57 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T21:57:29.365" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:56:59.565" grpc.time_ms=2.695 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1739vm-test-run-nixbot-gitlab> gitlab # [ 206.668982] gitaly[1387]: time="2026-06-29T21:56:59.661Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:09.270" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:56:59.651" grpc.time_ms=10.375 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1740vm-test-run-nixbot-gitlab> gitlab # [ 206.885845] nixbot[3185]: {"ts": "2026-06-29T21:56:59.877Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1741vm-test-run-nixbot-gitlab> gitlab # [ 206.906824] nixbot[3185]: {"ts": "2026-06-29T21:56:59.899Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1742vm-test-run-nixbot-gitlab> gitlab # [ 206.921275] nixbot[3185]: {"ts": "2026-06-29T21:56:59.913Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1743vm-test-run-nixbot-gitlab> gitlab # [ 206.932522] nixbot[3185]: {"ts": "2026-06-29T21:56:59.918Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1744vm-test-run-nixbot-gitlab> gitlab # [ 206.933915] nixbot[3185]: {"ts": "2026-06-29T21:56:59.921Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1745vm-test-run-nixbot-gitlab> gitlab # [ 206.949794] nixbot[3185]: {"ts": "2026-06-29T21:56:59.942Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1746vm-test-run-nixbot-gitlab> gitlab # [ 206.962643] nixbot[3185]: {"ts": "2026-06-29T21:56:59.955Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1747vm-test-run-nixbot-gitlab> gitlab # [ 206.973215] nixbot[3185]: {"ts": "2026-06-29T21:56:59.966Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1748vm-test-run-nixbot-gitlab> gitlab # [ 206.978575] nixbot[3185]: {"ts": "2026-06-29T21:56:59.971Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1749vm-test-run-nixbot-gitlab> gitlab # [ 206.983045] nixbot[3185]: {"ts": "2026-06-29T21:56:59.975Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1750vm-test-run-nixbot-gitlab> gitlab # [ 206.989731] nixbot[3185]: {"ts": "2026-06-29T21:56:59.982Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1751vm-test-run-nixbot-gitlab> gitlab # [ 207.000705] nixbot[3185]: {"ts": "2026-06-29T21:56:59.993Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1752vm-test-run-nixbot-gitlab> gitlab # [ 207.008400] nixbot[3185]: {"ts": "2026-06-29T21:57:00.001Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1753vm-test-run-nixbot-gitlab> gitlab # [ 207.015629] nixbot[3185]: {"ts": "2026-06-29T21:57:00.008Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1754vm-test-run-nixbot-gitlab> gitlab # [ 207.023900] nixbot[3185]: {"ts": "2026-06-29T21:57:00.016Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1755vm-test-run-nixbot-gitlab> gitlab # [ 207.030631] nixbot[3185]: {"ts": "2026-06-29T21:57:00.023Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1756vm-test-run-nixbot-gitlab> gitlab # [ 207.037941] nixbot[3185]: {"ts": "2026-06-29T21:57:00.030Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1757vm-test-run-nixbot-gitlab> gitlab # [ 207.040343] nixbot[3185]: {"ts": "2026-06-29T21:57:00.032Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1758vm-test-run-nixbot-gitlab> gitlab # [ 207.044423] nixbot[3185]: {"ts": "2026-06-29T21:57:00.037Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1759vm-test-run-nixbot-gitlab> gitlab # [ 207.349475] nixbot[3185]: INFO: Started server process [3185]1760vm-test-run-nixbot-gitlab> gitlab # [ 207.351909] nixbot[3185]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1761vm-test-run-nixbot-gitlab> gitlab # [ 207.355375] nixbot[3185]: INFO: ::ffff:127.0.0.1:45152 - "GET /health HTTP/1.1" 200 OK1762vm-test-run-nixbot-gitlab> gitlab # [ 207.362391] systemd[1]: Started nixbot CI service.1763vm-test-run-nixbot-gitlab> gitlab # [ 207.363750] systemd[1]: Reached target Multi-User System.1764vm-test-run-nixbot-gitlab> gitlab # [ 207.364821] systemd[1]: Startup finished in 1.347s (kernel) + 4.524s (initrd) + 3min 21.489s (userspace) = 3min 27.361s.1765vm-test-run-nixbot-gitlab> gitlab # [ 207.417600] gitaly[1387]: time="2026-06-29T21:57:00.410Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=970 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T21:57:30.195" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:00.396" grpc.time_ms=13.582 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1766vm-test-run-nixbot-gitlab> gitlab # [ 207.438681] gitaly[1387]: time="2026-06-29T21:57:00.431Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5CDSV6D4Y3E843986P0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T21:57:30.216" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:00.416" grpc.time_ms=14.915 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1767vm-test-run-nixbot-gitlab> gitlab # [ 207.475878] sidekiq-cluster[2173]: {"severity":"WARN","time":"2026-06-29T21:57:00.468Z","class":"CreatePipelineWorker","correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"ae941377e143a952c786905177b2065f7acae3f5","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"ae941377e143a952c786905177b2065f7acae3f5","ref":"refs/heads/master"},"retry":0}1768vm-test-run-nixbot-gitlab> gitlab # [ 207.481679] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:00.474Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a9557bc2a58f222f7058fcfb","created_at":"2026-06-29T21:56:58.647Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-a432a0aeb1ea4a40","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:66c681c6a4a2153af802b5ff028fba75d942bb28512196cdab47b736112f5f67","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.654Z","job_size_bytes":467,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a9557bc2a58f222f7058fcfb: done: 1.210296 sec","job_status":"done","queue_duration_s":0.60935,"scheduling_latency_s":0.609378,"gitaly_calls":7,"gitaly_duration_s":0.210634,"redis_calls":53,"redis_duration_s":0.116272,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.001727,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.107896,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004176,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.770737,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000339429,"duration_s":1.210296,"completed_at":"2026-06-29T21:57:00.474Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.086259,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1769vm-test-run-nixbot-gitlab> gitlab # [ 207.495232] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:00.476Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"eb2df00c264625b866c7ba98","created_at":"2026-06-29T21:56:58.895Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-d6d739d63cf2402b","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.899Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-eb2df00c264625b866c7ba98: start","job_status":"start","queue_duration_s":1.57615,"scheduling_latency_s":1.576172}1770vm-test-run-nixbot-gitlab> gitlab # [ 207.573361] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:00.513Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"eb2df00c264625b866c7ba98","created_at":"2026-06-29T21:56:58.895Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-d6d739d63cf2402b","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T21:56:58.899Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-eb2df00c264625b866c7ba98: done: 0.037339 sec","job_status":"done","queue_duration_s":1.57615,"scheduling_latency_s":1.576172,"redis_calls":19,"redis_duration_s":0.004201,"redis_read_bytes":21,"redis_write_bytes":4290,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000887,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000595,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2261,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001986,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000733,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006636039,"duration_s":0.037339,"completed_at":"2026-06-29T21:57:00.513Z","load_balancing_strategy":"primary","db_duration_s":0.00613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1771vm-test-run-nixbot-gitlab> gitlab # [ 207.649260] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:00.514Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"5b0e534fb53a29322667a8a4","created_at":"2026-06-29T21:57:00.502Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-6f410e75ebb841d3","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:00.506Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-5b0e534fb53a29322667a8a4: start","job_status":"start","queue_duration_s":0.00797,"scheduling_latency_s":0.007991}1772vm-test-run-nixbot-gitlab> gitlab # [ 207.650920] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:00.587Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"5b0e534fb53a29322667a8a4","created_at":"2026-06-29T21:57:00.502Z","trace_propagation_headers":{"sentry-trace":"536e190bc6dd48668882bd6902f36d3d-6f410e75ebb841d3","baggage":"sentry-trace_id=536e190bc6dd48668882bd6902f36d3d,sentry-release=Unknown"},"correlation_id":"01KWAP5CDSV6D4Y3E843986P0M","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:00.506Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-5b0e534fb53a29322667a8a4: done: 0.072708 sec","job_status":"done","queue_duration_s":0.00797,"scheduling_latency_s":0.007991,"redis_calls":11,"redis_duration_s":0.003713,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002693,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000597,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000617117,"duration_s":0.072708,"completed_at":"2026-06-29T21:57:00.587Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023357,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1773vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds)1774vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1775vm-test-run-nixbot-gitlab> gitlab # [ 207.859214] nixbot[3185]: INFO: ::ffff:127.0.0.1:45162 - "GET /health HTTP/1.1" 200 OK1776vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.05 seconds)1777vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1778vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1779vm-test-run-nixbot-gitlab> gitlab # [ 207.965669] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 661780vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1781vm-test-run-nixbot-gitlab> []1782vm-test-run-nixbot-gitlab> gitlab # [ 208.129816] gitaly[1387]: time="2026-06-29T21:57:01.122Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5F9VJ4RFKJGPCMG9CQ1M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T21:57:31.098" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:01.099" grpc.time_ms=23.23 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783vm-test-run-nixbot-gitlab> gitlab # [ 208.561828] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:01 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 12061784vm-test-run-nixbot-gitlab> gitlab # [ 208.574649] nixbot[3185]: {"ts": "2026-06-29T21:57:01.566Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1785vm-test-run-nixbot-gitlab> gitlab # [ 208.609944] nixbot[3185]: {"ts": "2026-06-29T21:57:01.602Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1786vm-test-run-nixbot-gitlab> gitlab # [ 208.723257] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1011787vm-test-run-nixbot-gitlab> gitlab # [ 208.733289] nixbot[3185]: {"ts": "2026-06-29T21:57:01.725Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1788vm-test-run-nixbot-gitlab> gitlab # [ 208.736080] nixbot[3185]: {"ts": "2026-06-29T21:57:01.728Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1789vm-test-run-nixbot-gitlab> gitlab # [ 208.874179] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1331790vm-test-run-nixbot-gitlab> gitlab # [ 208.883853] nixbot[3185]: {"ts": "2026-06-29T21:57:01.876Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1791vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1792vm-test-run-nixbot-gitlab> gitlab # [ 209.047538] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1361793vm-test-run-nixbot-gitlab> gitlab # [ 209.055875] nixbot[3185]: {"ts": "2026-06-29T21:57:02.048Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1794vm-test-run-nixbot-gitlab> gitlab # [ 209.122203] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 461795vm-test-run-nixbot-gitlab> gitlab # [ 209.129457] nixbot[3185]: {"ts": "2026-06-29T21:57:02.117Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1796vm-test-run-nixbot-gitlab> gitlab # [ 209.148402] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 1311797vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.18 seconds)1798vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-29T21:57:01.848Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1799vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.29 seconds)1800vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1801vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1802vm-test-run-nixbot-gitlab> gitlab # [ 209.180425] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 491803vm-test-run-nixbot-gitlab> gitlab # [ 209.181894] nixbot[3185]: {"ts": "2026-06-29T21:57:02.174Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1804vm-test-run-nixbot-gitlab> gitlab # [ 209.233288] gitaly[1387]: time="2026-06-29T21:57:02.223Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5H331MMZEBB5TQCW3ZRS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-29T21:57:32.015" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:02.215" grpc.time_ms=7.223 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805vm-test-run-nixbot-gitlab> gitlab # [ 209.240361] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 251806vm-test-run-nixbot-gitlab> gitlab # [ 209.268254] gitaly[1387]: time="2026-06-29T21:57:02.261Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5H331MMZEBB5TQCW3ZRS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:32.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:02.260" grpc.time_ms=0.708 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807vm-test-run-nixbot-gitlab> gitlab # [ 209.279965] gitaly[1387]: time="2026-06-29T21:57:02.273Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5H331MMZEBB5TQCW3ZRS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T21:57:12.265" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:02.266" grpc.time_ms=6.705 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808vm-test-run-nixbot-gitlab> gitlab # [ 209.323258] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1361809vm-test-run-nixbot-gitlab> gitlab # [ 209.324519] nixbot[3185]: {"ts": "2026-06-29T21:57:02.317Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1810vm-test-run-nixbot-gitlab> gitlab # [ 209.354106] gitaly[1387]: time="2026-06-29T21:57:02.347Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H4VW8QN00XW7J8ZA1K2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.336" grpc.time_ms=10.993 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1811vm-test-run-nixbot-gitlab> gitlab # [ 209.356936] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 1141812vm-test-run-nixbot-gitlab> gitlab # [ 209.385843] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 531813vm-test-run-nixbot-gitlab> gitlab # [ 209.386994] nixbot[3185]: {"ts": "2026-06-29T21:57:02.379Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1814vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1815vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1816vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1817vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1818vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done.1819vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1820vm-test-run-nixbot-gitlab> gitlab # [ 209.408210] nixbot[3185]: {"ts": "2026-06-29T21:57:02.401Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1821vm-test-run-nixbot-gitlab> gitlab # [ 209.502701] gitaly[1387]: time="2026-06-29T21:57:02.495Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5HB3BQ0ENV4280D6RQRA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.481" grpc.time_ms=13.539 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1822vm-test-run-nixbot-gitlab> gitlab # [ 209.504876] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 621823vm-test-run-nixbot-gitlab> gitlab # [ 209.619590] gitaly[1387]: time="2026-06-29T21:57:02.612Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HDV90K6TE0NBE3XPZZT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.601" pid=1387 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1824vm-test-run-nixbot-gitlab> gitlab # [ 209.620952] gitaly[1387]: time="2026-06-29T21:57:02.612Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HDV90K6TE0NBE3XPZZT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.601" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1825vm-test-run-nixbot-gitlab> gitlab # [ 209.622453] gitaly[1387]: time="2026-06-29T21:57:02.613Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HDV90K6TE0NBE3XPZZT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.601" grpc.time_ms=11.854 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1826vm-test-run-nixbot-gitlab> gitlab # [ 209.623811] gitlab-workhorse[1305]: time="2026-06-29T21:57:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491827vm-test-run-nixbot-gitlab> gitlab # [ 209.625121] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 901828vm-test-run-nixbot-gitlab> gitlab # [ 209.629712] gitaly[1387]: time="2026-06-29T21:57:02.621Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T21:57:12.009" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:02.610" grpc.time_ms=10.74 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1829vm-test-run-nixbot-gitlab> gitlab # [ 209.683596] gitaly[1387]: time="2026-06-29T21:57:02.676Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T21:57:32.663" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T21:57:02.664" grpc.time_ms=11.915 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1830vm-test-run-nixbot-gitlab> gitlab # [ 209.699108] gitaly[1387]: time="2026-06-29T21:57:02.691Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ duration_ms=125ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/allowed"1831vm-test-run-nixbot-gitlab> gitlab # [ 209.716386] gitaly[1387]: time="2026-06-29T21:57:02.709Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HGSMB1CZSCCJSWHCEAT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.686" pid=1387 remote_ip="2001:db8:1::1" request_sha=caeab738364be5526efcf73220096a4c24e3b376 response_bytes=378 span.kind=server system=grpc1832vm-test-run-nixbot-gitlab> gitlab # [ 209.719628] gitaly[1387]: time="2026-06-29T21:57:02.710Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1230 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HGSMB1CZSCCJSWHCEAT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.686" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1833vm-test-run-nixbot-gitlab> gitlab # [ 209.720855] gitlab-workhorse[1305]: time="2026-06-29T21:57:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3781834vm-test-run-nixbot-gitlab> gitlab # [ 209.722759] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 378 "" "git/2.54.0" 941835vm-test-run-nixbot-gitlab> gitlab # [ 209.723914] gitaly[1387]: time="2026-06-29T21:57:02.711Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1230 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HGSMB1CZSCCJSWHCEAT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.686" grpc.time_ms=24.671 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1836vm-test-run-nixbot-gitlab> gitlab # [ 209.771152] gitaly[1387]: time="2026-06-29T21:57:02.764Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ duration_ms=72ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/pre_receive"1837vm-test-run-nixbot-gitlab> gitlab # [ 209.772752] gitaly[1387]: time="2026-06-29T21:57:02.764Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5321 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:02.564" grpc.time_ms=199.819 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1838vm-test-run-nixbot-gitlab> gitlab # [ 209.850153] gitaly[1387]: time="2026-06-29T21:57:02.843Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:02.842" grpc.time_ms=0.579 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1839vm-test-run-nixbot-gitlab> gitlab # [ 209.904485] gitaly[1387]: time="2026-06-29T21:57:02.897Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:02.895" grpc.time_ms=1.234 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1840vm-test-run-nixbot-gitlab> gitlab # [ 209.938710] gitaly[1387]: time="2026-06-29T21:57:02.931Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5HRHG29K0PRRHBBW99CJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.913" grpc.time_ms=17.663 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1841vm-test-run-nixbot-gitlab> gitlab # [ 209.941676] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 681842vm-test-run-nixbot-gitlab> gitlab # [ 209.963116] gitaly[1387]: time="2026-06-29T21:57:02.955Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:02.954" grpc.time_ms=1.459 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1843vm-test-run-nixbot-gitlab> gitlab # [ 210.010796] gitaly[1387]: time="2026-06-29T21:57:03.003Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HTRP7FZWXRCPFEJA7ZE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.988" pid=1387 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1844vm-test-run-nixbot-gitlab> gitlab # [ 210.013430] gitaly[1387]: time="2026-06-29T21:57:03.004Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HTRP7FZWXRCPFEJA7ZE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.988" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1845vm-test-run-nixbot-gitlab> gitlab # [ 210.015537] gitlab-workhorse[1305]: time="2026-06-29T21:57:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671846vm-test-run-nixbot-gitlab> gitlab # [ 210.016976] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 691847vm-test-run-nixbot-gitlab> gitlab # [ 210.018852] gitaly[1387]: time="2026-06-29T21:57:03.005Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HTRP7FZWXRCPFEJA7ZE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.988" grpc.time_ms=16.381 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1848vm-test-run-nixbot-gitlab> gitlab # [ 210.078547] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:03.027Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","ae941377e143a952c786905177b2065f7acae3f5 8b5212701fe6acf76e74f205791f1e298c878a79 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"bd5b74a081aa2aca53a3fdd7","created_at":"2026-06-29T21:57:03.021Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-bea8952d85af4291","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.024Z","job_size_bytes":185,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-bd5b74a081aa2aca53a3fdd7: start","job_status":"start","queue_duration_s":0.002372,"scheduling_latency_s":0.002419}1849vm-test-run-nixbot-gitlab> gitlab # [ 210.080348] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:03.047Z","message":"changes: \"ae941377e143a952c786905177b2065f7acae3f5 8b5212701fe6acf76e74f205791f1e298c878a79 refs/heads/master\\n\""}1850vm-test-run-nixbot-gitlab> gitlab # [ 210.081406] gitaly[1387]: time="2026-06-29T21:57:03.063Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ duration_ms=77ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/post_receive"1851vm-test-run-nixbot-gitlab> gitlab # [ 210.109396] gitaly[1387]: time="2026-06-29T21:57:03.063Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4815 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:02.985" grpc.time_ms=78.321 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1852vm-test-run-nixbot-gitlab> gitlab # [ 210.110875] gitaly[1387]: time="2026-06-29T21:57:03.099Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:13.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:03.069" grpc.time_ms=30.292 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1853vm-test-run-nixbot-gitlab> gitlab # [ 210.123786] gitaly[1387]: time="2026-06-29T21:57:03.116Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T21:57:13.048" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:57:03.108" grpc.time_ms=7.95 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1854vm-test-run-nixbot-gitlab> gitlab # [ 210.186900] gitaly[1387]: time="2026-06-29T21:57:03.179Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HWZKYP57R4V5WFAEMZ0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:03.126" pid=1387 remote_ip="2001:db8:1::1" request_sha=6ab52ab0c8407de241e3cc58cb3b195614d825cb response_bytes=384 span.kind=server system=grpc1855vm-test-run-nixbot-gitlab> gitlab # [ 210.188235] gitaly[1387]: time="2026-06-29T21:57:03.180Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1210 command.oublock=0 command.real_time_ms=48 command.spawn_token_fork_ms=33 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HWZKYP57R4V5WFAEMZ0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:03.126" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1856vm-test-run-nixbot-gitlab> gitlab # [ 210.191173] gitaly[1387]: time="2026-06-29T21:57:03.183Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ diskcache=8eb4a1ea-5f16-47cd-9cf4-f97b9a1e5dce grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.477" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1857vm-test-run-nixbot-gitlab> gitlab # [ 210.192791] gitlab-workhorse[1305]: time="2026-06-29T21:57:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3841858vm-test-run-nixbot-gitlab> gitlab # [ 210.194075] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 384 "" "git/2.54.0" 1731859vm-test-run-nixbot-gitlab> gitlab # [ 210.195733] gitlab-workhorse[1305]: time="2026-06-29T21:57:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521860vm-test-run-nixbot-gitlab> gitlab # [ 210.196920] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:03 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 7861861vm-test-run-nixbot-gitlab> gitlab # [ 210.198148] gitaly[1387]: time="2026-06-29T21:57:03.183Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=274 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13114 command.oublock=32 command.real_time_ms=687 command.spawn_token_fork_ms=3 command.system_time_ms=225 command.user_time_ms=49 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:02.477" grpc.time_ms=705.439 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1862vm-test-run-nixbot-gitlab> gitlab # [ 210.204183] gitaly[1387]: time="2026-06-29T21:57:03.183Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1210 command.oublock=0 command.real_time_ms=48 command.spawn_token_fork_ms=33 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5HWZKYP57R4V5WFAEMZ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:03.126" grpc.time_ms=53.383 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1863vm-test-run-nixbot-gitlab> gitlab # [ 210.239466] gitaly[1387]: time="2026-06-29T21:57:03.231Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=8 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T21:57:13.118" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:03.179" grpc.time_ms=52.631 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1864vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1865vm-test-run-nixbot-gitlab> gitlab # ae94137..8b52127 master -> master1866vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.18 seconds)1867vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1868vm-test-run-nixbot-gitlab> gitlab # [ 210.333473] gitaly[1387]: time="2026-06-29T21:57:03.326Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-29T21:57:33.117" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:03.317" grpc.time_ms=8.565 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.03 seconds)1870vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8b5212701fe6acf76e74f205791f1e298c878a79'1871vm-test-run-nixbot-gitlab> gitlab # [ 210.371534] gitaly[1387]: time="2026-06-29T21:57:03.364Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T21:57:13.059" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T21:57:03.359" grpc.time_ms=4.258 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1872vm-test-run-nixbot-gitlab> gitlab # [ 210.383260] gitaly[1387]: time="2026-06-29T21:57:03.376Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:13.069" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:03.370" grpc.time_ms=5.745 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8b5212701fe6acf76e74f205791f1e298c878a79', in 0.07 seconds)1874vm-test-run-nixbot-gitlab> []1875vm-test-run-nixbot-gitlab> gitlab # [ 210.429714] nixbot[3185]: INFO: ::ffff:127.0.0.1:45178 - "GET /api/repos/gitlab/root/test-flake/builds?commit=8b5212701fe6acf76e74f205791f1e298c878a79 HTTP/1.1" 200 OK1876vm-test-run-nixbot-gitlab> gitlab # [ 210.449820] gitaly[1387]: time="2026-06-29T21:57:03.442Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:33.441" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:03.441" grpc.time_ms=0.709 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877vm-test-run-nixbot-gitlab> gitlab # [ 210.490350] gitaly[1387]: time="2026-06-29T21:57:03.483Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T21:57:33.469" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:03.470" grpc.time_ms=12.631 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878vm-test-run-nixbot-gitlab> gitlab # [ 210.514817] gitaly[1387]: time="2026-06-29T21:57:03.497Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:13.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:03.490" grpc.time_ms=7.037 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879vm-test-run-nixbot-gitlab> gitlab # [ 210.616742] gitaly[1387]: time="2026-06-29T21:57:03.609Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T21:57:13.006" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:03.607" grpc.time_ms=2.002 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880vm-test-run-nixbot-gitlab> gitlab # [ 210.646095] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:03.638Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","ae941377e143a952c786905177b2065f7acae3f5 8b5212701fe6acf76e74f205791f1e298c878a79 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"bd5b74a081aa2aca53a3fdd7","created_at":"2026-06-29T21:57:03.021Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-bea8952d85af4291","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.024Z","job_size_bytes":185,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-bd5b74a081aa2aca53a3fdd7: done: 0.610884 sec","job_status":"done","queue_duration_s":0.002372,"scheduling_latency_s":0.002419,"gitaly_calls":7,"gitaly_duration_s":0.242898,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.035709000000000005,"redis_read_bytes":350,"redis_write_bytes":13070,"redis_cache_calls":9,"redis_cache_duration_s":0.003246,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000853,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001892,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6770,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.00468,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.010531,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.014507,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.066,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.021,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.066,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.252198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001950528,"exclusive_lock_hold_duration_s":0.044558736,"duration_s":0.610884,"completed_at":"2026-06-29T21:57:03.638Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075495,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1881vm-test-run-nixbot-gitlab> gitlab # [ 210.668630] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:03.640Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8d9d6da879ac24388e4af761","created_at":"2026-06-29T21:57:03.119Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-5593c7f1bc2a42d3","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.121Z","job_size_bytes":46,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8d9d6da879ac24388e4af761: start","job_status":"start","queue_duration_s":0.519292,"scheduling_latency_s":0.51933}1882vm-test-run-nixbot-gitlab> gitlab # [ 210.694483] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:03.658Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8d9d6da879ac24388e4af761","created_at":"2026-06-29T21:57:03.119Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-5593c7f1bc2a42d3","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.121Z","job_size_bytes":46,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8d9d6da879ac24388e4af761: done: 0.018565 sec","job_status":"done","queue_duration_s":0.519292,"scheduling_latency_s":0.51933,"redis_calls":9,"redis_duration_s":0.002157,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000263,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018565,"completed_at":"2026-06-29T21:57:03.658Z","load_balancing_strategy":"primary","db_duration_s":0.001536,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1883vm-test-run-nixbot-gitlab> gitlab # [ 210.793223] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:03.660Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"986083cf1a5a23b021d0a6b9","created_at":"2026-06-29T21:57:03.170Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-5593c7f1bc2a42d3","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.173Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-986083cf1a5a23b021d0a6b9: start","job_status":"start","queue_duration_s":0.487118,"scheduling_latency_s":0.48714}1884vm-test-run-nixbot-gitlab> gitlab # [ 210.797539] gitaly[1387]: time="2026-06-29T21:57:03.790Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ diskcache=2c5b0b1a-06e0-4056-8302-e7953b808a7c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T03:57:03.675" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:03.676" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885vm-test-run-nixbot-gitlab> gitlab # [ 210.807383] gitaly[1387]: time="2026-06-29T21:57:03.790Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1645 command.oublock=0 command.real_time_ms=117 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-30T03:57:03.675" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:03.676" grpc.time_ms=114.535 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1886vm-test-run-nixbot-gitlab> gitlab # [ 210.832231] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:03.806Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"986083cf1a5a23b021d0a6b9","created_at":"2026-06-29T21:57:03.170Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-5593c7f1bc2a42d3","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.173Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-986083cf1a5a23b021d0a6b9: done: 0.145816 sec","job_status":"done","queue_duration_s":0.487118,"scheduling_latency_s":0.48714,"gitaly_calls":1,"gitaly_duration_s":0.117222,"redis_calls":12,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000593,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.129167255,"duration_s":0.145816,"completed_at":"2026-06-29T21:57:03.805Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1887vm-test-run-nixbot-gitlab> gitlab # [ 210.906477] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:03.808Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"95675fa2be1620524ecd8abd","created_at":"2026-06-29T21:57:03.433Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-5593c7f1bc2a42d3","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bbb27ae3365df3128be0963e56bdf02577c9fc1268c802fd9e1878912d12d2a4","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.435Z","job_size_bytes":467,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-95675fa2be1620524ecd8abd: start","job_status":"start","queue_duration_s":0.373524,"scheduling_latency_s":0.373546}1888vm-test-run-nixbot-gitlab> gitlab # [ 210.933971] gitaly[1387]: time="2026-06-29T21:57:03.832Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:33.628" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:03.829" grpc.time_ms=2.467 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889vm-test-run-nixbot-gitlab> gitlab # [ 210.954871] gitaly[1387]: time="2026-06-29T21:57:03.840Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T21:57:13.233" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:03.834" grpc.time_ms=6.66 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890vm-test-run-nixbot-gitlab> gitlab # [ 210.975932] gitaly[1387]: time="2026-06-29T21:57:03.867Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T21:57:33.666" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:03.866" grpc.time_ms=1.248 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1891vm-test-run-nixbot-gitlab> gitlab # [ 210.996418] gitaly[1387]: time="2026-06-29T21:57:03.875Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T21:57:33.670" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:03.870" grpc.time_ms=4.65 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892vm-test-run-nixbot-gitlab> gitlab # [ 211.023067] gitaly[1387]: time="2026-06-29T21:57:03.916Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:13.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:03.910" grpc.time_ms=6.328 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1893vm-test-run-nixbot-gitlab> gitlab # [ 211.140752] gitaly[1387]: time="2026-06-29T21:57:04.133Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T21:57:14.002" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:57:04.133" grpc.time_ms=0.279 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894vm-test-run-nixbot-gitlab> gitlab # [ 211.172393] gitaly[1387]: time="2026-06-29T21:57:04.165Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T21:57:14.027" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:04.158" grpc.time_ms=7.044 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1895vm-test-run-nixbot-gitlab> gitlab # [ 211.242213] gitaly[1387]: time="2026-06-29T21:57:04.234Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:04.234" grpc.time_ms=0.514 pid=1387 span.kind=server system=grpc1896vm-test-run-nixbot-gitlab> gitlab # [ 211.282641] gitaly[1387]: time="2026-06-29T21:57:04.275Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4160 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:04.275" grpc.time_ms=0.508 pid=1387 span.kind=server system=grpc1897vm-test-run-nixbot-gitlab> gitlab # [ 211.306376] gitaly[1387]: time="2026-06-29T21:57:04.299Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 diskcache=4b0ccfef-b050-468f-9c28-f69601c06afd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T21:57:14.045" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:57:04.186" pid=1387 ref=refs/keep-around/ae941377e143a952c786905177b2065f7acae3f5 remote_ip="2001:db8:1::1" revision=ae941377e143a952c786905177b2065f7acae3f5 span.kind=server system=grpc username=root1898vm-test-run-nixbot-gitlab> gitlab # [ 211.324952] gitaly[1387]: time="2026-06-29T21:57:04.317Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5285 command.oublock=8 command.real_time_ms=104 command.spawn_token_fork_ms=0 command.system_time_ms=51 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T21:57:14.045" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:57:04.186" grpc.time_ms=112.788 pid=1387 ref=refs/keep-around/ae941377e143a952c786905177b2065f7acae3f5 remote_ip="2001:db8:1::1" revision=ae941377e143a952c786905177b2065f7acae3f5 span.kind=server system=grpc username=root1899vm-test-run-nixbot-gitlab> gitlab # [ 211.350407] gitaly[1387]: time="2026-06-29T21:57:04.322Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:34.320" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:04.321" grpc.time_ms=0.414 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8b5212701fe6acf76e74f205791f1e298c878a79'1901vm-test-run-nixbot-gitlab> gitlab # [ 211.473360] gitaly[1387]: time="2026-06-29T21:57:04.466Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=965 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T21:57:34.248" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:04.448" grpc.time_ms=17.624 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1902vm-test-run-nixbot-gitlab> gitlab # [ 211.485660] nixbot[3185]: INFO: ::ffff:127.0.0.1:45186 - "GET /api/repos/gitlab/root/test-flake/builds?commit=8b5212701fe6acf76e74f205791f1e298c878a79 HTTP/1.1" 200 OK1903vm-test-run-nixbot-gitlab> gitlab # [ 211.497628] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae941377e143a952c786905177b2065f7acae3f5 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 11451904vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8b5212701fe6acf76e74f205791f1e298c878a79', in 0.08 seconds)1905vm-test-run-nixbot-gitlab> []1906vm-test-run-nixbot-gitlab> gitlab # [ 211.513794] nixbot[3185]: {"ts": "2026-06-29T21:57:04.505Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae941377e143a952c786905177b2065f7acae3f5 \"HTTP/1.1 201 Created\""}1907vm-test-run-nixbot-gitlab> gitlab # [ 211.522651] nixbot[3185]: {"ts": "2026-06-29T21:57:04.515Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-ae941377e143-3065aaec", "argv0": "bwrap"}1908vm-test-run-nixbot-gitlab> gitlab # [ 211.528668] gitaly[1387]: time="2026-06-29T21:57:04.521Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=46 command.spawn_token_fork_ms=41 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5H9XSWW44QEK9T6GH4WZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T21:57:34.270" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:04.471" grpc.time_ms=50.569 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909vm-test-run-nixbot-gitlab> gitlab # [ 211.550295] sidekiq-cluster[2173]: {"severity":"WARN","time":"2026-06-29T21:57:04.542Z","class":"CreatePipelineWorker","correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"ae941377e143a952c786905177b2065f7acae3f5","after":"8b5212701fe6acf76e74f205791f1e298c878a79","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"8b5212701fe6acf76e74f205791f1e298c878a79","ref":"refs/heads/master"},"retry":0}1910vm-test-run-nixbot-gitlab> gitlab # [ 211.578672] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:04.569Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"95675fa2be1620524ecd8abd","created_at":"2026-06-29T21:57:03.433Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-5593c7f1bc2a42d3","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bbb27ae3365df3128be0963e56bdf02577c9fc1268c802fd9e1878912d12d2a4","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.435Z","job_size_bytes":467,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-95675fa2be1620524ecd8abd: done: 0.761151 sec","job_status":"done","queue_duration_s":0.373524,"scheduling_latency_s":0.373546,"gitaly_calls":7,"gitaly_duration_s":0.107598,"redis_calls":33,"redis_duration_s":0.00793,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000862,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002967,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002323,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001778,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.175,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.17,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.417013,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.761151,"completed_at":"2026-06-29T21:57:04.569Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.174437,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1911vm-test-run-nixbot-gitlab> gitlab # [ 211.628313] systemd[1]: Started Nix Daemon.1912vm-test-run-nixbot-gitlab> gitlab # [ 211.659375] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:04.651Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"22713e45-4a4b-49d1-a831-b72dc2d59a26\"}"],"class":"WebHookWorker","jid":"a97c0eb79f39ccfc0cc3da4b","created_at":"2026-06-29T21:57:03.580Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-5593c7f1bc2a42d3","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3481e28122ae44ca34f6da1b12b8216db6b1da10791c3295ec0f5c0a4ee17a51","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.587Z","job_size_bytes":1749,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a97c0eb79f39ccfc0cc3da4b: start","job_status":"start","queue_duration_s":1.064118,"scheduling_latency_s":1.06416}1913vm-test-run-nixbot-gitlab> gitlab # [ 211.720530] nix-daemon[3466]: accepted connection from pid 3465, user nixbot1914vm-test-run-nixbot-gitlab> gitlab # [ 211.726782] nix-daemon[3466]: accepted connection from pid 3467, user nixbot1915vm-test-run-nixbot-gitlab> gitlab # [ 211.833772] nixbot[3185]: INFO: ::ffff:127.0.0.1:45194 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1916vm-test-run-nixbot-gitlab> gitlab # [ 211.893544] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:04.884Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"22713e45-4a4b-49d1-a831-b72dc2d59a26\"}"],"class":"WebHookWorker","jid":"a97c0eb79f39ccfc0cc3da4b","created_at":"2026-06-29T21:57:03.580Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-5593c7f1bc2a42d3","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3481e28122ae44ca34f6da1b12b8216db6b1da10791c3295ec0f5c0a4ee17a51","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:03.587Z","job_size_bytes":1749,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a97c0eb79f39ccfc0cc3da4b: done: 0.23306 sec","job_status":"done","queue_duration_s":1.064118,"scheduling_latency_s":1.06416,"redis_calls":19,"redis_duration_s":0.042176,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003318,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.014799,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.005541,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.018518,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.02280429499998604,"cpu_s":0.086295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036066,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.23306,"completed_at":"2026-06-29T21:57:04.884Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020405,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1917vm-test-run-nixbot-gitlab> gitlab # [ 211.992320] gitaly[1387]: time="2026-06-29T21:57:04.907Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5KPX69X6K92941MGZFN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:04.899" grpc.time_ms=8.19 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1918vm-test-run-nixbot-gitlab> gitlab # [ 211.993473] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:04 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 481919vm-test-run-nixbot-gitlab> gitlab # [ 212.027265] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:05.019Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"3d4b27f516074b278c9b2413","created_at":"2026-06-29T21:57:04.327Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-40fad66218ed4a17","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:04.334Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-3d4b27f516074b278c9b2413: start","job_status":"start","queue_duration_s":0.685659,"scheduling_latency_s":0.685696}1920vm-test-run-nixbot-gitlab> gitlab # [ 212.086253] gitaly[1387]: time="2026-06-29T21:57:05.079Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5KRFJ38GBGGYH3ZD04BK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:05.068" pid=1387 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1921vm-test-run-nixbot-gitlab> gitlab # [ 212.087828] gitaly[1387]: time="2026-06-29T21:57:05.079Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5KRFJ38GBGGYH3ZD04BK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:05.068" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1922vm-test-run-nixbot-gitlab> gitlab # [ 212.088938] gitlab-workhorse[1305]: time="2026-06-29T21:57:05Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671923vm-test-run-nixbot-gitlab> gitlab # [ 212.096493] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:05 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 1691924vm-test-run-nixbot-gitlab> gitlab # [ 212.101358] gitaly[1387]: time="2026-06-29T21:57:05.081Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5KRFJ38GBGGYH3ZD04BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:05.068" grpc.time_ms=11.393 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1925vm-test-run-nixbot-gitlab> gitlab # [ 212.184346] nixbot[3185]: {"ts": "2026-06-29T21:57:05.177Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1926vm-test-run-nixbot-gitlab> gitlab # [ 212.197663] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:05.190Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"3d4b27f516074b278c9b2413","created_at":"2026-06-29T21:57:04.327Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-40fad66218ed4a17","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:04.334Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-3d4b27f516074b278c9b2413: done: 0.17033 sec","job_status":"done","queue_duration_s":0.685659,"scheduling_latency_s":0.685696,"redis_calls":18,"redis_duration_s":0.027195,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.010499,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001493206,"duration_s":0.17033,"completed_at":"2026-06-29T21:57:05.190Z","load_balancing_strategy":"primary","db_duration_s":0.046837,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1927vm-test-run-nixbot-gitlab> gitlab # [ 212.201506] nix-daemon[3472]: unexpected Nix daemon error: error: interrupted by the user1928vm-test-run-nixbot-gitlab> gitlab # [ 212.206422] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:05.199Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"aaeccd520344f76e8a1b7a41","created_at":"2026-06-29T21:57:04.334Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-40fad66218ed4a17","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:04.338Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-aaeccd520344f76e8a1b7a41: start","job_status":"start","queue_duration_s":0.860865,"scheduling_latency_s":0.860896}1929vm-test-run-nixbot-gitlab> gitlab # [ 212.238470] gitaly[1387]: time="2026-06-29T21:57:05.231Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:35.229" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:05.230" grpc.time_ms=0.775 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930vm-test-run-nixbot-gitlab> gitlab # [ 212.317362] gitaly[1387]: time="2026-06-29T21:57:05.309Z" level=info msg="finished unary call with code OK" catfile.duration_ms=36 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=36 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5M1Q1PBFWRHN34MZ5SZG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:35.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:05.272" grpc.time_ms=37.583 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931vm-test-run-nixbot-gitlab> gitlab # [ 212.349080] gitaly[1387]: time="2026-06-29T21:57:05.342Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=94 command.spawn_token_fork_ms=67 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T21:57:35.245" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:05.246" grpc.time_ms=95.669 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932vm-test-run-nixbot-gitlab> gitlab # [ 212.382436] gitaly[1387]: time="2026-06-29T21:57:05.375Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:15.236" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:05.367" grpc.time_ms=8.179 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933vm-test-run-nixbot-gitlab> gitlab # [ 212.404815] gitaly[1387]: time="2026-06-29T21:57:05.397Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T21:57:15.266" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:57:05.397" grpc.time_ms=0.215 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1934vm-test-run-nixbot-gitlab> gitlab # [ 212.427174] gitaly[1387]: time="2026-06-29T21:57:05.420Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T21:57:15.281" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:05.412" grpc.time_ms=7.399 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8b5212701fe6acf76e74f205791f1e298c878a79'1936vm-test-run-nixbot-gitlab> gitlab # [ 212.443704] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:05.436Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"aaeccd520344f76e8a1b7a41","created_at":"2026-06-29T21:57:04.334Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-40fad66218ed4a17","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:04.338Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-aaeccd520344f76e8a1b7a41: done: 0.23732 sec","job_status":"done","queue_duration_s":0.860865,"scheduling_latency_s":0.860896,"redis_calls":15,"redis_duration_s":0.017489,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001575,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003458,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.012456,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.093,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.067,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.09408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001453257,"duration_s":0.23732,"completed_at":"2026-06-29T21:57:05.436Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.096951,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1937vm-test-run-nixbot-gitlab> gitlab # [ 212.451071] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:05.442Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c960f89dfa4b3401434d7de8","created_at":"2026-06-29T21:57:04.464Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-40fad66218ed4a17","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:04.471Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c960f89dfa4b3401434d7de8: start","job_status":"start","queue_duration_s":0.970748,"scheduling_latency_s":0.970828}1938vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8b5212701fe6acf76e74f205791f1e298c878a79', in 0.06 seconds)1939vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-06-29T21:57:05.167059Z', 'started_at': None, 'finished_at': None}]1940vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.42 seconds)1941vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses1942vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses1943vm-test-run-nixbot-gitlab> gitlab # [ 212.574602] gitaly[1387]: time="2026-06-29T21:57:05.563Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5M1Q1PBFWRHN34MZ5SZG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:15.123" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:05.553" grpc.time_ms=9.055 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1944vm-test-run-nixbot-gitlab> gitlab # [ 212.597653] nixbot[3185]: INFO: ::ffff:127.0.0.1:45210 - "GET /api/repos/gitlab/root/test-flake/builds?commit=8b5212701fe6acf76e74f205791f1e298c878a79 HTTP/1.1" 200 OK1945vm-test-run-nixbot-gitlab> gitlab # [ 212.600966] gitaly[1387]: time="2026-06-29T21:57:05.587Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:05.586" grpc.time_ms=0.657 pid=1387 span.kind=server system=grpc1946vm-test-run-nixbot-gitlab> gitlab # [ 212.644110] gitaly[1387]: time="2026-06-29T21:57:05.633Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4160 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:05.631" grpc.time_ms=0.936 pid=1387 span.kind=server system=grpc1947vm-test-run-nixbot-gitlab> gitlab # [ 212.651781] gitaly[1387]: time="2026-06-29T21:57:05.644Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H diskcache=aec9dcd0-201c-4620-a175-923ed5592800 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T21:57:15.059" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:57:05.539" pid=1387 ref=refs/keep-around/8b5212701fe6acf76e74f205791f1e298c878a79 remote_ip="2001:db8:1::1" revision=8b5212701fe6acf76e74f205791f1e298c878a79 span.kind=server system=grpc username=root1948vm-test-run-nixbot-gitlab> gitlab # [ 212.653370] gitaly[1387]: time="2026-06-29T21:57:05.644Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5160 command.oublock=8 command.real_time_ms=95 command.spawn_token_fork_ms=4 command.system_time_ms=41 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T21:57:15.059" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T21:57:05.539" grpc.time_ms=105.024 pid=1387 ref=refs/keep-around/8b5212701fe6acf76e74f205791f1e298c878a79 remote_ip="2001:db8:1::1" revision=8b5212701fe6acf76e74f205791f1e298c878a79 span.kind=server system=grpc username=root1949vm-test-run-nixbot-gitlab> gitlab # [ 212.661572] gitaly[1387]: time="2026-06-29T21:57:05.654Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:35.653" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:05.653" grpc.time_ms=0.529 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1950vm-test-run-nixbot-gitlab> gitlab # [ 212.728928] gitaly[1387]: time="2026-06-29T21:57:05.721Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=7 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=7 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T21:57:35.513" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:05.713" grpc.time_ms=7.67 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1951vm-test-run-nixbot-gitlab> gitlab # [ 212.733973] gitaly[1387]: time="2026-06-29T21:57:05.727Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5MDYJPZ9E3B6G1FEVS1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:35.667" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:05.667" grpc.time_ms=58.957 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1952vm-test-run-nixbot-gitlab> gitlab # [ 212.753176] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae941377e143a952c786905177b2065f7acae3f5 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 5371953vm-test-run-nixbot-gitlab> gitlab # [ 212.754791] nixbot[3185]: {"ts": "2026-06-29T21:57:05.747Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae941377e143a952c786905177b2065f7acae3f5 \"HTTP/1.1 201 Created\""}1954vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 0.22 seconds)1955vm-test-run-nixbot-gitlab> []1956vm-test-run-nixbot-gitlab> gitlab # [ 212.797141] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 1911957vm-test-run-nixbot-gitlab> gitlab # [ 212.806853] gitaly[1387]: time="2026-06-29T21:57:05.799Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5MJR8316H2WE50A4AG1F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:35.597" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:05.798" grpc.time_ms=1.231 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1958vm-test-run-nixbot-gitlab> gitlab # [ 212.817217] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8b5212701fe6acf76e74f205791f1e298c878a79 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 6271959vm-test-run-nixbot-gitlab> gitlab # [ 212.818670] nixbot[3185]: {"ts": "2026-06-29T21:57:05.808Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8b5212701fe6acf76e74f205791f1e298c878a79 \"HTTP/1.1 201 Created\""}1960vm-test-run-nixbot-gitlab> gitlab # [ 212.825889] nixbot[3185]: {"ts": "2026-06-29T21:57:05.818Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-8b5212701fe6-91789c6f", "argv0": "bwrap"}1961vm-test-run-nixbot-gitlab> gitlab # [ 212.846299] gitaly[1387]: time="2026-06-29T21:57:05.839Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5MJR8316H2WE50A4AG1F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:15.099" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:05.829" grpc.time_ms=9.466 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962vm-test-run-nixbot-gitlab> gitlab # [ 212.876750] gitaly[1387]: time="2026-06-29T21:57:05.869Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:35.667" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:05.868" grpc.time_ms=1.164 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963vm-test-run-nixbot-gitlab> gitlab # [ 212.896640] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae941377e143a952c786905177b2065f7acae3f5 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1331964vm-test-run-nixbot-gitlab> gitlab # [ 212.899040] nixbot[3185]: {"ts": "2026-06-29T21:57:05.891Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae941377e143a952c786905177b2065f7acae3f5 \"HTTP/1.1 201 Created\""}1965vm-test-run-nixbot-gitlab> gitlab # [ 212.948152] gitaly[1387]: time="2026-06-29T21:57:05.938Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:35.737" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:05.937" grpc.time_ms=0.662 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966vm-test-run-nixbot-gitlab> gitlab # [ 212.960314] nix-daemon[3466]: accepted connection from pid 3549, user nixbot1967vm-test-run-nixbot-gitlab> gitlab # [ 212.964467] nix-daemon[3466]: accepted connection from pid 3550, user nixbot1968vm-test-run-nixbot-gitlab> gitlab # [ 212.980589] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:05.972Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6d553a2a0590d2cf05bc7303","created_at":"2026-06-29T21:57:05.969Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-f0319222b43f493a","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"c960f89dfa4b3401434d7de8","job_size_bytes":3,"pid":2173,"job_status":"deduplicated","message":"PipelineProcessWorker JID-6d553a2a0590d2cf05bc7303: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1969vm-test-run-nixbot-gitlab> gitlab # [ 213.011931] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.004Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c960f89dfa4b3401434d7de8","created_at":"2026-06-29T21:57:04.464Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-40fad66218ed4a17","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:04.471Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2173,"job_status":"rescheduled","message":"PipelineProcessWorker JID-c960f89dfa4b3401434d7de8: rescheduled"}1970vm-test-run-nixbot-gitlab> gitlab # [ 213.064320] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.056Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c960f89dfa4b3401434d7de8","created_at":"2026-06-29T21:57:04.464Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-40fad66218ed4a17","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:04.471Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c960f89dfa4b3401434d7de8: done: 0.614256 sec","job_status":"done","queue_duration_s":0.970748,"scheduling_latency_s":0.970828,"gitaly_calls":3,"gitaly_duration_s":0.073124,"redis_calls":54,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.115444,"redis_read_bytes":292,"redis_write_bytes":8548,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.014775,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.021935,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.009217,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":4,"redis_queues_duration_s":0.031162,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2218,"redis_queues_metadata_calls":12,"redis_queues_metadata_duration_s":0.015827,"redis_queues_metadata_read_bytes":89,"redis_queues_metadata_write_bytes":1500,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.002375,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.001151,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004227,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.201,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.017,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.096,"db_ci_duration_s":0.105,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.134549,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.44295099,"duration_s":0.614256,"completed_at":"2026-06-29T21:57:06.056Z","load_balancing_strategy":"primary","db_duration_s":0.23769,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1971vm-test-run-nixbot-gitlab> gitlab # [ 213.159224] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.151Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a97c0eb79f39ccfc0cc3da4b"],"class":"WebHooks::LogExecutionWorker","jid":"18286a92ccaa9ed103819a64","created_at":"2026-06-29T21:57:04.837Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-ddf516e3886344af","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2587cfa7f79e79d98f18edde34d756d930404e5f1c48eb47c5ccb78da449e28d","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:04.860Z","job_size_bytes":2507,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-18286a92ccaa9ed103819a64: start","job_status":"start","queue_duration_s":1.290359,"scheduling_latency_s":1.290429}1972vm-test-run-nixbot-gitlab> gitlab # [ 213.384004] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.375Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a97c0eb79f39ccfc0cc3da4b"],"class":"WebHooks::LogExecutionWorker","jid":"18286a92ccaa9ed103819a64","created_at":"2026-06-29T21:57:04.837Z","trace_propagation_headers":{"sentry-trace":"cc5f7c929e924d0dad5793f6fc8224ac-ddf516e3886344af","baggage":"sentry-trace_id=cc5f7c929e924d0dad5793f6fc8224ac,sentry-release=Unknown"},"correlation_id":"01KWAP5H9XSWW44QEK9T6GH4WZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2587cfa7f79e79d98f18edde34d756d930404e5f1c48eb47c5ccb78da449e28d","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:04.860Z","job_size_bytes":2507,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-18286a92ccaa9ed103819a64: done: 0.225124 sec","job_status":"done","queue_duration_s":1.290359,"scheduling_latency_s":1.290429,"redis_calls":10,"redis_duration_s":0.029072999999999998,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006848,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015482,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006743,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.03,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.03,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.125018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001393194,"duration_s":0.225124,"completed_at":"2026-06-29T21:57:06.375Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973vm-test-run-nixbot-gitlab> gitlab # [ 213.488444] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.480Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"3cd05e017f3bcbacbe56f3b6","created_at":"2026-06-29T21:57:05.658Z","trace_propagation_headers":{"sentry-trace":"73fbe3c859494a9d898be07ed6d68683-3a8c2826ee6e479f","baggage":"sentry-trace_id=73fbe3c859494a9d898be07ed6d68683,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5M0WEPHB9T32580CQC3H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:05.660Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-3cd05e017f3bcbacbe56f3b6: start","job_status":"start","queue_duration_s":0.8195,"scheduling_latency_s":0.81957}1974vm-test-run-nixbot-gitlab> gitlab # [ 213.591483] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.583Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"3cd05e017f3bcbacbe56f3b6","created_at":"2026-06-29T21:57:05.658Z","trace_propagation_headers":{"sentry-trace":"73fbe3c859494a9d898be07ed6d68683-3a8c2826ee6e479f","baggage":"sentry-trace_id=73fbe3c859494a9d898be07ed6d68683,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5M0WEPHB9T32580CQC3H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:05.660Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-3cd05e017f3bcbacbe56f3b6: done: 0.103421 sec","job_status":"done","queue_duration_s":0.8195,"scheduling_latency_s":0.81957,"redis_calls":7,"redis_duration_s":0.018294,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.013374,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00492,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.103421,"completed_at":"2026-06-29T21:57:06.583Z","load_balancing_strategy":"primary","db_duration_s":0.013244,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1975vm-test-run-nixbot-gitlab> gitlab # [ 213.605480] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.597Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"a83fec3fd29beda7c57d5a83","created_at":"2026-06-29T21:57:05.652Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-f0319222b43f493a","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:05.664Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a83fec3fd29beda7c57d5a83: start","job_status":"start","queue_duration_s":0.933538,"scheduling_latency_s":0.933609}1976vm-test-run-nixbot-gitlab> gitlab # [ 213.728726] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.720Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"a83fec3fd29beda7c57d5a83","created_at":"2026-06-29T21:57:05.652Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-f0319222b43f493a","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:05.664Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a83fec3fd29beda7c57d5a83: done: 0.123134 sec","job_status":"done","queue_duration_s":0.933538,"scheduling_latency_s":0.933609,"redis_calls":10,"redis_duration_s":0.021830000000000002,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003105,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005415,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001749384,"duration_s":0.123134,"completed_at":"2026-06-29T21:57:06.720Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1977vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses1978vm-test-run-nixbot-gitlab> gitlab # [ 213.740676] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.732Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3360a7814c404e52631a0a21","created_at":"2026-06-29T21:57:05.661Z","trace_propagation_headers":{"sentry-trace":"73fbe3c859494a9d898be07ed6d68683-3a8c2826ee6e479f","baggage":"sentry-trace_id=73fbe3c859494a9d898be07ed6d68683,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5M0WEPHB9T32580CQC3H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:05.666Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3360a7814c404e52631a0a21: start","job_status":"start","queue_duration_s":1.065933,"scheduling_latency_s":1.065986}1979vm-test-run-nixbot-gitlab> gitlab # [ 213.877449] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.870Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3360a7814c404e52631a0a21","created_at":"2026-06-29T21:57:05.661Z","trace_propagation_headers":{"sentry-trace":"73fbe3c859494a9d898be07ed6d68683-3a8c2826ee6e479f","baggage":"sentry-trace_id=73fbe3c859494a9d898be07ed6d68683,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5M0WEPHB9T32580CQC3H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:05.666Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3360a7814c404e52631a0a21: done: 0.137246 sec","job_status":"done","queue_duration_s":1.065933,"scheduling_latency_s":1.065986,"redis_calls":10,"redis_duration_s":0.015641,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.007038,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.137246,"completed_at":"2026-06-29T21:57:06.869Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.054366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1980vm-test-run-nixbot-gitlab> gitlab # [ 213.914358] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:06.877Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"fd4dbf7821fc5006680a16bf","created_at":"2026-06-29T21:57:05.793Z","trace_propagation_headers":{"sentry-trace":"73fbe3c859494a9d898be07ed6d68683-3a8c2826ee6e479f","baggage":"sentry-trace_id=73fbe3c859494a9d898be07ed6d68683,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5M0WEPHB9T32580CQC3H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:05.796Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fd4dbf7821fc5006680a16bf: start","job_status":"start","queue_duration_s":1.080884,"scheduling_latency_s":1.080921}1981vm-test-run-nixbot-gitlab> gitlab # [ 213.916397] gitaly[1387]: time="2026-06-29T21:57:06.906Z" level=info msg="finished unary call with code OK" catfile.duration_ms=23 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=23 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5NMQ4RE3GHTMG9RV8DGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:36.180" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:06.881" grpc.time_ms=24.134 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982vm-test-run-nixbot-gitlab> gitlab # [ 213.995307] gitaly[1387]: time="2026-06-29T21:57:06.988Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T21:57:36.786" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:06.987" grpc.time_ms=0.793 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983vm-test-run-nixbot-gitlab> gitlab # [ 214.015409] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1681984vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 0.22 seconds)1985vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T21:57:05.776Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1986vm-test-run-nixbot-gitlab> gitlab # [ 214.065139] gitaly[1387]: time="2026-06-29T21:57:07.058Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:37.056" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:07.057" grpc.time_ms=0.776 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987vm-test-run-nixbot-gitlab> gitlab # [ 214.218786] gitaly[1387]: time="2026-06-29T21:57:07.211Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5M0WEPHB9T32580CQC3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:37.208" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:07.209" grpc.time_ms=1.365 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988vm-test-run-nixbot-gitlab> gitlab # [ 214.349365] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:07.340Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"fd4dbf7821fc5006680a16bf","created_at":"2026-06-29T21:57:05.793Z","trace_propagation_headers":{"sentry-trace":"73fbe3c859494a9d898be07ed6d68683-3a8c2826ee6e479f","baggage":"sentry-trace_id=73fbe3c859494a9d898be07ed6d68683,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5M0WEPHB9T32580CQC3H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:05.796Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fd4dbf7821fc5006680a16bf: done: 0.463238 sec","job_status":"done","queue_duration_s":1.080884,"scheduling_latency_s":1.080921,"gitaly_calls":3,"gitaly_duration_s":0.105961,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.049984,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.004118,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.009901,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000551,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.020913,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001153,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.002306,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006924,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.14,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.086,"db_ci_duration_s":0.054,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.113822,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.348242863,"duration_s":0.463238,"completed_at":"2026-06-29T21:57:07.340Z","load_balancing_strategy":"primary","db_duration_s":0.143935,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1989vm-test-run-nixbot-gitlab> gitlab # [ 214.371798] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:07.355Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f013c3f5bdfc44fe305d85cc","created_at":"2026-06-29T21:57:06.006Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-f0319222b43f493a","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:06.009Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f013c3f5bdfc44fe305d85cc: start","job_status":"start","queue_duration_s":1.345623,"scheduling_latency_s":1.345696}1990vm-test-run-nixbot-gitlab> gitlab # [ 214.957611] gitaly[1387]: time="2026-06-29T21:57:07.949Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:07.948" grpc.time_ms=0.671 pid=1387 span.kind=server system=grpc1991vm-test-run-nixbot-gitlab> gitlab # [ 215.003375] gitaly[1387]: time="2026-06-29T21:57:07.996Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:07.995" grpc.time_ms=0.481 pid=1387 span.kind=server system=grpc1992vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses1993vm-test-run-nixbot-gitlab> gitlab # [ 215.047614] gitaly[1387]: time="2026-06-29T21:57:08.040Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:08.039" grpc.time_ms=0.828 pid=1387 span.kind=server system=grpc1994vm-test-run-nixbot-gitlab> gitlab # [ 215.072359] gitaly[1387]: time="2026-06-29T21:57:08.065Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 diskcache=022f951e-d281-44c4-8fab-88b5c7c66fce grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:37.897" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:07.898" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1995vm-test-run-nixbot-gitlab> gitlab # [ 215.088295] gitaly[1387]: time="2026-06-29T21:57:08.081Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=139 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7062 command.oublock=0 command.real_time_ms=159 command.spawn_token_fork_ms=1 command.system_time_ms=104 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:37.897" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:07.898" grpc.time_ms=167.346 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1996vm-test-run-nixbot-gitlab> gitlab # [ 215.109778] gitaly[1387]: time="2026-06-29T21:57:08.102Z" level=info msg="finished unary call with code OK" catfile.duration_ms=18 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=18 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5PT7YM7GFHYTMRKTJTCA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:38.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:08.082" grpc.time_ms=19.354 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1997vm-test-run-nixbot-gitlab> gitlab # [ 215.131102] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 831998vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 0.11 seconds)1999vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T21:57:05.776Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2000vm-test-run-nixbot-gitlab> gitlab # [ 215.263697] gitaly[1387]: time="2026-06-29T21:57:08.256Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T21:57:38.255" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:08.255" grpc.time_ms=0.968 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2001vm-test-run-nixbot-gitlab> gitlab # [ 215.360383] gitaly[1387]: time="2026-06-29T21:57:08.353Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:38.349" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:08.350" grpc.time_ms=1.873 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2002vm-test-run-nixbot-gitlab> gitlab # [ 215.484302] gitaly[1387]: time="2026-06-29T21:57:08.477Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5J7G3TSHWZ5DZBTNEWE7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:38.473" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:08.475" grpc.time_ms=1.622 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2003vm-test-run-nixbot-gitlab> gitlab # [ 215.531777] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:08.522Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f013c3f5bdfc44fe305d85cc","created_at":"2026-06-29T21:57:06.006Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-f0319222b43f493a","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:06.009Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f013c3f5bdfc44fe305d85cc: done: 1.167422 sec","job_status":"done","queue_duration_s":1.345623,"scheduling_latency_s":1.345696,"gitaly_calls":4,"gitaly_duration_s":0.24917,"redis_calls":98,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.197072,"redis_read_bytes":188,"redis_write_bytes":22259,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.015566,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.010535,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.036528,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":20,"redis_queues_duration_s":0.012108,"redis_queues_read_bytes":30,"redis_queues_write_bytes":11298,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.033709,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.003719,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002973,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.066368,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.307,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.127,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.174,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.105,"db_ci_duration_s":0.202,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.288032,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":1.002520991,"duration_s":1.167422,"completed_at":"2026-06-29T21:57:08.522Z","load_balancing_strategy":"primary","db_duration_s":0.419407,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2004vm-test-run-nixbot-gitlab> gitlab # [ 215.634649] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:08.626Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"e356301019f0ff9f3d6007d0","created_at":"2026-06-29T21:57:06.967Z","trace_propagation_headers":{"sentry-trace":"73fbe3c859494a9d898be07ed6d68683-47b8999773a5451a","baggage":"sentry-trace_id=73fbe3c859494a9d898be07ed6d68683,sentry-release=Unknown"},"correlation_id":"01KWAP5M0WEPHB9T32580CQC3H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:06.971Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e356301019f0ff9f3d6007d0: start","job_status":"start","queue_duration_s":1.654932,"scheduling_latency_s":1.655001}2005vm-test-run-nixbot-gitlab> gitlab # [ 215.760562] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:08.752Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"e356301019f0ff9f3d6007d0","created_at":"2026-06-29T21:57:06.967Z","trace_propagation_headers":{"sentry-trace":"73fbe3c859494a9d898be07ed6d68683-47b8999773a5451a","baggage":"sentry-trace_id=73fbe3c859494a9d898be07ed6d68683,sentry-release=Unknown"},"correlation_id":"01KWAP5M0WEPHB9T32580CQC3H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:06.971Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e356301019f0ff9f3d6007d0: done: 0.125876 sec","job_status":"done","queue_duration_s":1.654932,"scheduling_latency_s":1.655001,"redis_calls":7,"redis_duration_s":0.016699,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012194,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004505,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.125876,"completed_at":"2026-06-29T21:57:08.752Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2006vm-test-run-nixbot-gitlab> gitlab # [ 215.775203] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:08.767Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"443578db282fa4392dcc06bb","created_at":"2026-06-29T21:57:07.087Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20da67ec613aee78fd14c10988a6a8d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.098Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-443578db282fa4392dcc06bb: start","job_status":"start","queue_duration_s":1.66854,"scheduling_latency_s":1.6686}2007vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses2008vm-test-run-nixbot-gitlab> gitlab # [ 216.226817] gitaly[1387]: time="2026-06-29T21:57:09.218Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5QXN3QRQNXP0109G5G57 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:39.216" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:09.217" grpc.time_ms=1.408 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2009vm-test-run-nixbot-gitlab> gitlab # [ 216.269779] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 872010vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 0.14 seconds)2011vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T21:57:05.776Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2012vm-test-run-nixbot-gitlab> gitlab # [ 216.322324] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.314Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"443578db282fa4392dcc06bb","created_at":"2026-06-29T21:57:07.087Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20da67ec613aee78fd14c10988a6a8d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.098Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-443578db282fa4392dcc06bb: done: 0.547745 sec","job_status":"done","queue_duration_s":1.66854,"scheduling_latency_s":1.6686,"redis_calls":9,"redis_duration_s":0.016007,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009675,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006332,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":123,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":123,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.111,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.111,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.300293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001530642,"exclusive_lock_hold_duration_s":0.492741116,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.547745,"completed_at":"2026-06-29T21:57:09.314Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.231576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2013vm-test-run-nixbot-gitlab> gitlab # [ 216.403326] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"eab6475c9fb8018c2316228a","created_at":"2026-06-29T21:57:07.110Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2de95c6c76a1a4220a5de373e7498bbe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T21:57:07.121Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-eab6475c9fb8018c2316228a: start","job_status":"start","queue_duration_s":2.274202,"scheduling_latency_s":2.274286}2014vm-test-run-nixbot-gitlab> gitlab # [ 216.450254] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.442Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2015vm-test-run-nixbot-gitlab> gitlab # [ 216.466391] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.458Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"eab6475c9fb8018c2316228a","created_at":"2026-06-29T21:57:07.110Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2de95c6c76a1a4220a5de373e7498bbe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T21:57:07.121Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-eab6475c9fb8018c2316228a: done: 0.06343 sec","job_status":"done","queue_duration_s":2.274202,"scheduling_latency_s":2.274286,"redis_calls":9,"redis_duration_s":0.008399,"redis_read_bytes":7,"redis_write_bytes":1213,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002074,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005343,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06343,"completed_at":"2026-06-29T21:57:09.458Z","load_balancing_strategy":"primary","db_duration_s":0.001433,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2016vm-test-run-nixbot-gitlab> gitlab # [ 216.534486] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.526Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ccc4cf1a19c83674e2748f4e","created_at":"2026-06-29T21:57:07.133Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3011342af05f95922cab1c249074336","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.139Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ccc4cf1a19c83674e2748f4e: start","job_status":"start","queue_duration_s":2.386617,"scheduling_latency_s":2.386687}2017vm-test-run-nixbot-gitlab> gitlab # [ 216.586222] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.578Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ccc4cf1a19c83674e2748f4e","created_at":"2026-06-29T21:57:07.133Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3011342af05f95922cab1c249074336","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.139Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ccc4cf1a19c83674e2748f4e: done: 0.051685 sec","job_status":"done","queue_duration_s":2.386617,"scheduling_latency_s":2.386687,"redis_calls":9,"redis_duration_s":0.005993999999999999,"redis_read_bytes":7,"redis_write_bytes":1080,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00133,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":369,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003325,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001339,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051685,"completed_at":"2026-06-29T21:57:09.578Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2018vm-test-run-nixbot-gitlab> gitlab # [ 216.589730] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"034380eea12fb8b6053b0be9","created_at":"2026-06-29T21:57:07.152Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7043668f70d8013520cf545bbdf0720","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.158Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-034380eea12fb8b6053b0be9: start","job_status":"start","queue_duration_s":2.423445,"scheduling_latency_s":2.423913}2019vm-test-run-nixbot-gitlab> gitlab # [ 216.620267] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.612Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"034380eea12fb8b6053b0be9","created_at":"2026-06-29T21:57:07.152Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7043668f70d8013520cf545bbdf0720","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.158Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-034380eea12fb8b6053b0be9: done: 0.031151 sec","job_status":"done","queue_duration_s":2.423445,"scheduling_latency_s":2.423913,"redis_calls":11,"redis_duration_s":0.0055839999999999996,"redis_read_bytes":9,"redis_write_bytes":1450,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002217,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":674,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000854,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031151,"completed_at":"2026-06-29T21:57:09.612Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2020vm-test-run-nixbot-gitlab> gitlab # [ 216.677120] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8ff22d1e7701dff64a4e5eb5","created_at":"2026-06-29T21:57:07.192Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3929839ae0fdb3e558904aa24c55e6de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.201Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8ff22d1e7701dff64a4e5eb5: start","job_status":"start","queue_duration_s":2.412825,"scheduling_latency_s":2.412846}2021vm-test-run-nixbot-gitlab> gitlab # [ 216.700617] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8ff22d1e7701dff64a4e5eb5","created_at":"2026-06-29T21:57:07.192Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3929839ae0fdb3e558904aa24c55e6de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.201Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8ff22d1e7701dff64a4e5eb5: done: 0.017448 sec","job_status":"done","queue_duration_s":2.412825,"scheduling_latency_s":2.412846,"redis_calls":9,"redis_duration_s":0.004056,"redis_read_bytes":7,"redis_write_bytes":1246,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000857,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":415,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002271,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017448,"completed_at":"2026-06-29T21:57:09.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000677,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2022vm-test-run-nixbot-gitlab> gitlab # [ 216.805405] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.634Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"bf77e2da4a0f0a27da99009a","created_at":"2026-06-29T21:57:07.220Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d172e8de527463182b30ff6fcecb95e0","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.225Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-bf77e2da4a0f0a27da99009a: start","job_status":"start","queue_duration_s":2.40864,"scheduling_latency_s":2.408663}2023vm-test-run-nixbot-gitlab> gitlab # [ 216.807205] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.654Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"bf77e2da4a0f0a27da99009a","created_at":"2026-06-29T21:57:07.220Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d172e8de527463182b30ff6fcecb95e0","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.225Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-bf77e2da4a0f0a27da99009a: done: 0.020211 sec","job_status":"done","queue_duration_s":2.40864,"scheduling_latency_s":2.408663,"redis_calls":11,"redis_duration_s":0.004798,"redis_read_bytes":9,"redis_write_bytes":1472,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00085,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":385,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002165,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001783,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000909334,"exclusive_lock_hold_duration_s":0.002571835,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.020211,"completed_at":"2026-06-29T21:57:09.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024vm-test-run-nixbot-gitlab> gitlab # [ 216.861944] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.656Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2152f1065622909b1dcbcb89","created_at":"2026-06-29T21:57:07.238Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02194f857978fbb5eed1f2f252cac8d7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.241Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2152f1065622909b1dcbcb89: start","job_status":"start","queue_duration_s":2.414511,"scheduling_latency_s":2.414533}2025vm-test-run-nixbot-gitlab> gitlab # [ 216.934492] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2152f1065622909b1dcbcb89","created_at":"2026-06-29T21:57:07.238Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02194f857978fbb5eed1f2f252cac8d7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.241Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2152f1065622909b1dcbcb89: done: 0.020193 sec","job_status":"done","queue_duration_s":2.414511,"scheduling_latency_s":2.414533,"redis_calls":11,"redis_duration_s":0.004872,"redis_read_bytes":9,"redis_write_bytes":1453,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000871,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":381,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002123,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001878,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000598959,"exclusive_lock_hold_duration_s":0.002508699,"duration_s":0.020193,"completed_at":"2026-06-29T21:57:09.676Z","load_balancing_strategy":"primary","db_duration_s":0.002091,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026vm-test-run-nixbot-gitlab> gitlab # [ 216.953213] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"47314eab172393ff8dd59666","created_at":"2026-06-29T21:57:07.250Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32ae647dc51255e96764452fc2ba8a50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.255Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-47314eab172393ff8dd59666: start","job_status":"start","queue_duration_s":2.422857,"scheduling_latency_s":2.422877}2027vm-test-run-nixbot-gitlab> gitlab # [ 217.027922] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.710Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"47314eab172393ff8dd59666","created_at":"2026-06-29T21:57:07.250Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32ae647dc51255e96764452fc2ba8a50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.255Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-47314eab172393ff8dd59666: done: 0.031792 sec","job_status":"done","queue_duration_s":2.422857,"scheduling_latency_s":2.422877,"redis_calls":11,"redis_duration_s":0.004231,"redis_read_bytes":9,"redis_write_bytes":1563,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000848,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":401,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001928,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000685282,"exclusive_lock_hold_duration_s":0.014148726,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.031792,"completed_at":"2026-06-29T21:57:09.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2028vm-test-run-nixbot-gitlab> gitlab # [ 217.101972] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3a5e6b5b04b2b6552caadc54","created_at":"2026-06-29T21:57:07.279Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94a96c452858ae46d875549b4abee46c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.287Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3a5e6b5b04b2b6552caadc54: start","job_status":"start","queue_duration_s":2.424343,"scheduling_latency_s":2.424364}2029vm-test-run-nixbot-gitlab> gitlab # [ 217.103537] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.721Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2030vm-test-run-nixbot-gitlab> gitlab # [ 217.105368] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3a5e6b5b04b2b6552caadc54","created_at":"2026-06-29T21:57:07.279Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94a96c452858ae46d875549b4abee46c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.287Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3a5e6b5b04b2b6552caadc54: done: 0.014896 sec","job_status":"done","queue_duration_s":2.424343,"scheduling_latency_s":2.424364,"redis_calls":9,"redis_duration_s":0.002422,"redis_read_bytes":7,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":413,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001463,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014896,"completed_at":"2026-06-29T21:57:09.726Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000493,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2031vm-test-run-nixbot-gitlab> gitlab # [ 217.149856] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ab7ce1f2664175b2552980a6","created_at":"2026-06-29T21:57:07.321Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b5ce1ded6cff6e67c4b857b930c6a1ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T21:57:07.333Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ab7ce1f2664175b2552980a6: start","job_status":"start","queue_duration_s":2.394105,"scheduling_latency_s":2.394128}2032vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses2033vm-test-run-nixbot-gitlab> gitlab # [ 217.172340] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.753Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ab7ce1f2664175b2552980a6","created_at":"2026-06-29T21:57:07.321Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b5ce1ded6cff6e67c4b857b930c6a1ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T21:57:07.333Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ab7ce1f2664175b2552980a6: done: 0.025897 sec","job_status":"done","queue_duration_s":2.394105,"scheduling_latency_s":2.394128,"redis_calls":9,"redis_duration_s":0.00303,"redis_read_bytes":7,"redis_write_bytes":1122,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000487,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":381,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000871,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025897,"completed_at":"2026-06-29T21:57:09.753Z","load_balancing_strategy":"primary","db_duration_s":0.001845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2034vm-test-run-nixbot-gitlab> gitlab # [ 217.254576] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.755Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c6fde69853b5ba51c8de4875","created_at":"2026-06-29T21:57:07.365Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e641f12c953ceae17a6697b15b88a89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T21:57:07.376Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c6fde69853b5ba51c8de4875: start","job_status":"start","queue_duration_s":2.378404,"scheduling_latency_s":2.378485}2035vm-test-run-nixbot-gitlab> gitlab # [ 217.259661] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.765Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2036vm-test-run-nixbot-gitlab> gitlab # [ 217.266255] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.770Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c6fde69853b5ba51c8de4875","created_at":"2026-06-29T21:57:07.365Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e641f12c953ceae17a6697b15b88a89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T21:57:07.376Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c6fde69853b5ba51c8de4875: done: 0.015287 sec","job_status":"done","queue_duration_s":2.378404,"scheduling_latency_s":2.378485,"redis_calls":9,"redis_duration_s":0.002149,"redis_read_bytes":7,"redis_write_bytes":1206,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":405,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015287,"completed_at":"2026-06-29T21:57:09.770Z","load_balancing_strategy":"primary","db_duration_s":0.000581,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2037vm-test-run-nixbot-gitlab> gitlab # [ 217.333981] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fe50c96f5a51efaef65b1e6b","created_at":"2026-06-29T21:57:07.393Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66180ca4ce32701ee6c9335324f4ccfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.398Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fe50c96f5a51efaef65b1e6b: start","job_status":"start","queue_duration_s":2.373168,"scheduling_latency_s":2.373189}2038vm-test-run-nixbot-gitlab> gitlab # [ 217.357854] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.781Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2039vm-test-run-nixbot-gitlab> gitlab # [ 217.363857] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.786Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fe50c96f5a51efaef65b1e6b","created_at":"2026-06-29T21:57:07.393Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66180ca4ce32701ee6c9335324f4ccfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.398Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fe50c96f5a51efaef65b1e6b: done: 0.014453 sec","job_status":"done","queue_duration_s":2.373168,"scheduling_latency_s":2.373189,"redis_calls":9,"redis_duration_s":0.00209,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":393,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014453,"completed_at":"2026-06-29T21:57:09.786Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2040vm-test-run-nixbot-gitlab> gitlab # [ 217.476450] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.787Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"565891eb6ec299e75b501001","created_at":"2026-06-29T21:57:07.412Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ceca3c665e9d44ee2b497652c5b6f56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.419Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-565891eb6ec299e75b501001: start","job_status":"start","queue_duration_s":2.368084,"scheduling_latency_s":2.368218}2041vm-test-run-nixbot-gitlab> gitlab # [ 217.478214] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.821Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"565891eb6ec299e75b501001","created_at":"2026-06-29T21:57:07.412Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ceca3c665e9d44ee2b497652c5b6f56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.419Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-565891eb6ec299e75b501001: done: 0.03398 sec","job_status":"done","queue_duration_s":2.368084,"scheduling_latency_s":2.368218,"redis_calls":11,"redis_duration_s":0.003654,"redis_read_bytes":9,"redis_write_bytes":1572,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":403,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001398,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000710984,"exclusive_lock_hold_duration_s":0.011865195,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.03398,"completed_at":"2026-06-29T21:57:09.821Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2042vm-test-run-nixbot-gitlab> gitlab # [ 217.528120] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.823Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f698e6d9a76b791a7bb4b7e3","created_at":"2026-06-29T21:57:07.443Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34442c15067f2026dddf7d02d6756a58","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.451Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f698e6d9a76b791a7bb4b7e3: start","job_status":"start","queue_duration_s":2.372106,"scheduling_latency_s":2.372128}2043vm-test-run-nixbot-gitlab> gitlab # [ 217.551673] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f698e6d9a76b791a7bb4b7e3","created_at":"2026-06-29T21:57:07.443Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34442c15067f2026dddf7d02d6756a58","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.451Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f698e6d9a76b791a7bb4b7e3: done: 0.02616 sec","job_status":"done","queue_duration_s":2.372106,"scheduling_latency_s":2.372128,"redis_calls":9,"redis_duration_s":0.002983,"redis_read_bytes":7,"redis_write_bytes":1237,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001081,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":411,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.02616,"completed_at":"2026-06-29T21:57:09.849Z","load_balancing_strategy":"primary","db_duration_s":0.00526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2044vm-test-run-nixbot-gitlab> gitlab # [ 217.623051] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:09.851Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0c70a1c16169982a10a50793","created_at":"2026-06-29T21:57:07.469Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8658997ef0c80070f926748b4dea093a","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.475Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0c70a1c16169982a10a50793: start","job_status":"start","queue_duration_s":2.375664,"scheduling_latency_s":2.375693}2045vm-test-run-nixbot-gitlab> gitlab # [ 217.693505] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.282Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0c70a1c16169982a10a50793","created_at":"2026-06-29T21:57:07.469Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8658997ef0c80070f926748b4dea093a","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.475Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0c70a1c16169982a10a50793: done: 0.431532 sec","job_status":"done","queue_duration_s":2.375664,"scheduling_latency_s":2.375693,"redis_calls":573,"redis_duration_s":0.151673,"redis_read_bytes":571,"redis_write_bytes":49844,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.149941,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.325629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.431532,"completed_at":"2026-06-29T21:57:10.282Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2046vm-test-run-nixbot-gitlab> gitlab # [ 217.712391] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.284Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"432e92c8dde2a33f077cfb08","created_at":"2026-06-29T21:57:07.492Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.503Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-432e92c8dde2a33f077cfb08: start","job_status":"start","queue_duration_s":2.780505,"scheduling_latency_s":2.780534}2047vm-test-run-nixbot-gitlab> gitlab # [ 217.734602] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.371Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"432e92c8dde2a33f077cfb08","created_at":"2026-06-29T21:57:07.492Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.503Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-432e92c8dde2a33f077cfb08: done: 0.087177 sec","job_status":"done","queue_duration_s":2.780505,"scheduling_latency_s":2.780534,"redis_calls":62,"redis_duration_s":0.0161,"redis_read_bytes":110,"redis_write_bytes":41444,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":385,"redis_queues_calls":2,"redis_queues_duration_s":0.000661,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.014189,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000759,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.087177,"completed_at":"2026-06-29T21:57:10.371Z","load_balancing_strategy":"primary","db_duration_s":0.000693,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2048vm-test-run-nixbot-gitlab> gitlab # [ 217.846111] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f4af0d8859cf7f9786dcf510","created_at":"2026-06-29T21:57:07.528Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fb89847a91f2ab11f24913736295ec8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.554Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f4af0d8859cf7f9786dcf510: start","job_status":"start","queue_duration_s":2.818239,"scheduling_latency_s":2.818269}2049vm-test-run-nixbot-gitlab> gitlab # [ 217.847348] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f4af0d8859cf7f9786dcf510","created_at":"2026-06-29T21:57:07.528Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fb89847a91f2ab11f24913736295ec8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.554Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f4af0d8859cf7f9786dcf510: done: 0.023668 sec","job_status":"done","queue_duration_s":2.818239,"scheduling_latency_s":2.818269,"redis_calls":12,"redis_duration_s":0.004531,"redis_read_bytes":10,"redis_write_bytes":1630,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000727,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":391,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002228,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001576,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003695721,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.023668,"completed_at":"2026-06-29T21:57:10.396Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003566,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2050vm-test-run-nixbot-gitlab> gitlab # [ 217.895876] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.398Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d5c6aecb02bcf9b80bcddfae","created_at":"2026-06-29T21:57:07.589Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6830d6c5b3ff18068337acc49f14de36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.598Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d5c6aecb02bcf9b80bcddfae: start","job_status":"start","queue_duration_s":2.799443,"scheduling_latency_s":2.799465}2051vm-test-run-nixbot-gitlab> gitlab # [ 217.919515] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.409Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2052vm-test-run-nixbot-gitlab> gitlab # [ 217.926992] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.414Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d5c6aecb02bcf9b80bcddfae","created_at":"2026-06-29T21:57:07.589Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6830d6c5b3ff18068337acc49f14de36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.598Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d5c6aecb02bcf9b80bcddfae: done: 0.016096 sec","job_status":"done","queue_duration_s":2.799443,"scheduling_latency_s":2.799465,"redis_calls":9,"redis_duration_s":0.003373,"redis_read_bytes":7,"redis_write_bytes":1232,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000793,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":411,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00066,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016096,"completed_at":"2026-06-29T21:57:10.414Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053vm-test-run-nixbot-gitlab> gitlab # [ 217.994984] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"e6f1e2878895bcb313737239","created_at":"2026-06-29T21:57:07.641Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.649Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-e6f1e2878895bcb313737239: start","job_status":"start","queue_duration_s":2.766397,"scheduling_latency_s":2.766419}2054vm-test-run-nixbot-gitlab> gitlab # [ 218.016465] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"e6f1e2878895bcb313737239","created_at":"2026-06-29T21:57:07.641Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.649Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-e6f1e2878895bcb313737239: done: 0.025065 sec","job_status":"done","queue_duration_s":2.766397,"scheduling_latency_s":2.766419,"redis_calls":16,"redis_duration_s":0.0052049999999999996,"redis_read_bytes":18,"redis_write_bytes":4401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000813,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":371,"redis_queues_calls":2,"redis_queues_duration_s":0.000339,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002858,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025065,"completed_at":"2026-06-29T21:57:10.441Z","load_balancing_strategy":"primary","db_duration_s":0.001609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2055vm-test-run-nixbot-gitlab> gitlab # [ 218.086338] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"741d5ea0037aa317c5961dcc","created_at":"2026-06-29T21:57:07.704Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43a02a0840ada426b45a15e2041126c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.715Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-741d5ea0037aa317c5961dcc: start","job_status":"start","queue_duration_s":2.727242,"scheduling_latency_s":2.727263}2056vm-test-run-nixbot-gitlab> gitlab # [ 218.109841] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.452Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2057vm-test-run-nixbot-gitlab> gitlab # [ 218.115294] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"741d5ea0037aa317c5961dcc","created_at":"2026-06-29T21:57:07.704Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43a02a0840ada426b45a15e2041126c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.715Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-741d5ea0037aa317c5961dcc: done: 0.014812 sec","job_status":"done","queue_duration_s":2.727242,"scheduling_latency_s":2.727263,"redis_calls":9,"redis_duration_s":0.002566,"redis_read_bytes":7,"redis_write_bytes":1166,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000484,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":395,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000706,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014812,"completed_at":"2026-06-29T21:57:10.457Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2058vm-test-run-nixbot-gitlab> gitlab # [ 218.183481] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.459Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"bc58c8d6da3aa61e4820a134","created_at":"2026-06-29T21:57:07.764Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ff820a00b6dbad9eaa45c1cda6416dd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.772Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-bc58c8d6da3aa61e4820a134: start","job_status":"start","queue_duration_s":2.686764,"scheduling_latency_s":2.686829}2059vm-test-run-nixbot-gitlab> gitlab # [ 218.206376] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"bc58c8d6da3aa61e4820a134","created_at":"2026-06-29T21:57:07.764Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-3ca1dc50dcdd4a19","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ff820a00b6dbad9eaa45c1cda6416dd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.772Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-bc58c8d6da3aa61e4820a134: done: 0.018227 sec","job_status":"done","queue_duration_s":2.686764,"scheduling_latency_s":2.686829,"redis_calls":9,"redis_duration_s":0.004,"redis_read_bytes":7,"redis_write_bytes":1038,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000712,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002458,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00083,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018227,"completed_at":"2026-06-29T21:57:10.477Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2060vm-test-run-nixbot-gitlab> gitlab # [ 218.275525] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.479Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3a31b6c3069abd3eebe14c44","created_at":"2026-06-29T21:57:07.834Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.856Z","job_size_bytes":23,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3a31b6c3069abd3eebe14c44: start","job_status":"start","queue_duration_s":2.623244,"scheduling_latency_s":2.623278}2061vm-test-run-nixbot-gitlab> gitlab # [ 218.302165] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.604Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3a31b6c3069abd3eebe14c44","created_at":"2026-06-29T21:57:07.834Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:07.856Z","job_size_bytes":23,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3a31b6c3069abd3eebe14c44: done: 0.125522 sec","job_status":"done","queue_duration_s":2.623244,"scheduling_latency_s":2.623278,"redis_calls":15,"redis_duration_s":0.005361,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000934,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000336,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002006,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001191,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093965,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349207,"duration_s":0.125522,"completed_at":"2026-06-29T21:57:10.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019049,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2062vm-test-run-nixbot-gitlab> gitlab # [ 218.380741] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.606Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"eba27cdac30a31cf2f9718d0","created_at":"2026-06-29T21:57:10.595Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-7a2a447c10cd4bdb","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:e18e4c7da2e5f6be32df3196b6a7bb07a28af60e89ac5a99900d17c7df49d3b2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.598Z","job_size_bytes":464,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-eba27cdac30a31cf2f9718d0: start","job_status":"start","queue_duration_s":0.008547,"scheduling_latency_s":0.008571}2063vm-test-run-nixbot-gitlab> gitlab # [ 218.407679] nix-daemon[3466]: accepted connection from pid 3607, user nixbot2064vm-test-run-nixbot-gitlab> gitlab # [ 218.409423] gitaly[1387]: time="2026-06-29T21:57:11.262Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:41.261" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:11.261" grpc.time_ms=0.823 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2065vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 1.34 seconds)2066vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T21:57:05.776Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2067vm-test-run-nixbot-gitlab> gitlab # [ 218.429236] sidekiq-cluster[2173]: {"severity":"WARN","time":"2026-06-29T21:57:10.777Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"eba27cdac30a31cf2f9718d0","created_at":"2026-06-29T21:57:10.595Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-7a2a447c10cd4bdb","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:e18e4c7da2e5f6be32df3196b6a7bb07a28af60e89ac5a99900d17c7df49d3b2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.598Z","job_size_bytes":464,"pid":2173,"sidekiq_tid":"m2w9","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-eba27cdac30a31cf2f9718d0: fail: 0.17052 sec","job_status":"fail","queue_duration_s":0.008547,"scheduling_latency_s":0.008571,"redis_calls":7,"redis_duration_s":0.001951,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000601,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.14,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.139,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.17052,"completed_at":"2026-06-29T21:57:10.777Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.14011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2068vm-test-run-nixbot-gitlab> gitlab # [ 219.049144] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:10.781Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c58c44290a90873ad0ecc5cd","created_at":"2026-06-29T21:57:08.086Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.096Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c58c44290a90873ad0ecc5cd: start","job_status":"start","queue_duration_s":2.684365,"scheduling_latency_s":2.684405}2069vm-test-run-nixbot-gitlab> gitlab # [ 219.075293] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.064Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c58c44290a90873ad0ecc5cd","created_at":"2026-06-29T21:57:08.086Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.096Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c58c44290a90873ad0ecc5cd: done: 0.283151 sec","job_status":"done","queue_duration_s":2.684365,"scheduling_latency_s":2.684405,"redis_calls":10,"redis_duration_s":0.0046029999999999995,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003436,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000525,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.191,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.126,"db_ci_duration_s":0.065,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000420724,"duration_s":0.283151,"completed_at":"2026-06-29T21:57:11.064Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.191161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070vm-test-run-nixbot-gitlab> gitlab # [ 219.179949] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.067Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f6916f3d3c522fd1ac3ec4db","created_at":"2026-06-29T21:57:08.164Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.183Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f6916f3d3c522fd1ac3ec4db: start","job_status":"start","queue_duration_s":2.883875,"scheduling_latency_s":2.883918}2071vm-test-run-nixbot-gitlab> gitlab # [ 219.181308] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.230Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f6916f3d3c522fd1ac3ec4db","created_at":"2026-06-29T21:57:08.164Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.183Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f6916f3d3c522fd1ac3ec4db: done: 0.162999 sec","job_status":"done","queue_duration_s":2.883875,"scheduling_latency_s":2.883918,"redis_calls":10,"redis_duration_s":0.0031190000000000002,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000459,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001619,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.13,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.129,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000429943,"duration_s":0.162999,"completed_at":"2026-06-29T21:57:11.230Z","load_balancing_strategy":"primary","db_duration_s":0.12974,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072vm-test-run-nixbot-gitlab> gitlab # [ 219.244441] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.232Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4e447df9f25193ba686b0000","created_at":"2026-06-29T21:57:08.200Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.209Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4e447df9f25193ba686b0000: start","job_status":"start","queue_duration_s":3.023024,"scheduling_latency_s":3.023049}2073vm-test-run-nixbot-gitlab> gitlab # [ 219.326360] gitaly[1387]: time="2026-06-29T21:57:11.572Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5T7FX7FWV2CG547FRK1Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:41.570" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:11.570" grpc.time_ms=0.977 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2074vm-test-run-nixbot-gitlab> gitlab # [ 219.328307] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 562075vm-test-run-nixbot-gitlab> gitlab # [ 219.329347] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.260Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4e447df9f25193ba686b0000","created_at":"2026-06-29T21:57:08.200Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.209Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4e447df9f25193ba686b0000: done: 0.027974 sec","job_status":"done","queue_duration_s":3.023024,"scheduling_latency_s":3.023049,"redis_calls":13,"redis_duration_s":0.003167,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000512,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001395,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000688,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000572,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349486,"duration_s":0.027974,"completed_at":"2026-06-29T21:57:11.260Z","load_balancing_strategy":"primary","db_duration_s":0.003085,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076vm-test-run-nixbot-gitlab> gitlab # [ 219.366485] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.261Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"65bd8d62e90dd4f25f0eabf9","created_at":"2026-06-29T21:57:08.210Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.217Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-65bd8d62e90dd4f25f0eabf9: start","job_status":"start","queue_duration_s":3.043952,"scheduling_latency_s":3.043973}2077vm-test-run-nixbot-gitlab> gitlab # [ 219.393337] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.285Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"65bd8d62e90dd4f25f0eabf9","created_at":"2026-06-29T21:57:08.210Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.217Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-65bd8d62e90dd4f25f0eabf9: done: 0.023916 sec","job_status":"done","queue_duration_s":3.043952,"scheduling_latency_s":3.043973,"redis_calls":10,"redis_duration_s":0.002484,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00053,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001506,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000390273,"duration_s":0.023916,"completed_at":"2026-06-29T21:57:11.285Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078vm-test-run-nixbot-gitlab> gitlab # [ 219.472206] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.287Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"3b3c49f1ffc904304ead34af","created_at":"2026-06-29T21:57:08.218Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.226Z","job_size_bytes":19,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-3b3c49f1ffc904304ead34af: start","job_status":"start","queue_duration_s":3.060343,"scheduling_latency_s":3.060364}2079vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses2080vm-test-run-nixbot-gitlab> gitlab # [ 219.489348] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.312Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"3b3c49f1ffc904304ead34af","created_at":"2026-06-29T21:57:08.218Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.226Z","job_size_bytes":19,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-3b3c49f1ffc904304ead34af: done: 0.025113 sec","job_status":"done","queue_duration_s":3.060343,"scheduling_latency_s":3.060364,"redis_calls":11,"redis_duration_s":0.00281,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000652,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001715,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000402286,"duration_s":0.025113,"completed_at":"2026-06-29T21:57:11.312Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002866,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081vm-test-run-nixbot-gitlab> gitlab # [ 219.610291] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.313Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"07afc7b07b10564fc5bc5b66","created_at":"2026-06-29T21:57:08.229Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.236Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-07afc7b07b10564fc5bc5b66: start","job_status":"start","queue_duration_s":3.077035,"scheduling_latency_s":3.077057}2082vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 0.11 seconds)2083vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T21:57:05.776Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2084vm-test-run-nixbot-gitlab> gitlab # [ 219.612184] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.336Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"07afc7b07b10564fc5bc5b66","created_at":"2026-06-29T21:57:08.229Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.236Z","job_size_bytes":65,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-07afc7b07b10564fc5bc5b66: done: 0.023392 sec","job_status":"done","queue_duration_s":3.077035,"scheduling_latency_s":3.077057,"redis_calls":10,"redis_duration_s":0.002338,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001447,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000347531,"duration_s":0.023392,"completed_at":"2026-06-29T21:57:11.336Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085vm-test-run-nixbot-gitlab> gitlab # [ 219.661220] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.338Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"34a7bd0eee0b7494030440ff","created_at":"2026-06-29T21:57:08.239Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.240Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-34a7bd0eee0b7494030440ff: start","job_status":"start","queue_duration_s":3.097968,"scheduling_latency_s":3.097989}2086vm-test-run-nixbot-gitlab> gitlab # [ 219.689426] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.363Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"34a7bd0eee0b7494030440ff","created_at":"2026-06-29T21:57:08.239Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.240Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-34a7bd0eee0b7494030440ff: done: 0.024325 sec","job_status":"done","queue_duration_s":3.097968,"scheduling_latency_s":3.097989,"redis_calls":7,"redis_duration_s":0.0016909999999999998,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024325,"completed_at":"2026-06-29T21:57:11.363Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087vm-test-run-nixbot-gitlab> gitlab # [ 219.766999] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.364Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ddebd376c22c2266931f27f6","created_at":"2026-06-29T21:57:08.354Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.376Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ddebd376c22c2266931f27f6: start","job_status":"start","queue_duration_s":2.987972,"scheduling_latency_s":2.987993}2088vm-test-run-nixbot-gitlab> gitlab # [ 219.783042] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.393Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ddebd376c22c2266931f27f6","created_at":"2026-06-29T21:57:08.354Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.376Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ddebd376c22c2266931f27f6: done: 0.028731 sec","job_status":"done","queue_duration_s":2.987972,"scheduling_latency_s":2.987993,"redis_calls":10,"redis_duration_s":0.002359,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001413,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000504,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346413,"duration_s":0.028731,"completed_at":"2026-06-29T21:57:11.393Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2089vm-test-run-nixbot-gitlab> gitlab # [ 219.911934] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.394Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7d10d35e92ef382680cba181","created_at":"2026-06-29T21:57:08.380Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.399Z","job_size_bytes":6,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7d10d35e92ef382680cba181: start","job_status":"start","queue_duration_s":2.995199,"scheduling_latency_s":2.99522}2090vm-test-run-nixbot-gitlab> gitlab # [ 219.913210] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.415Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7d10d35e92ef382680cba181","created_at":"2026-06-29T21:57:08.380Z","trace_propagation_headers":{"sentry-trace":"e0237c2aedcf461f8691639f2210df74-a15281fe84134e04","baggage":"sentry-trace_id=e0237c2aedcf461f8691639f2210df74,sentry-release=Unknown"},"correlation_id":"01KWAP5J7G3TSHWZ5DZBTNEWE7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:08.399Z","job_size_bytes":6,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7d10d35e92ef382680cba181: done: 0.020391 sec","job_status":"done","queue_duration_s":2.995199,"scheduling_latency_s":2.99522,"redis_calls":10,"redis_duration_s":0.002541,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000585,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352,"duration_s":0.020391,"completed_at":"2026-06-29T21:57:11.415Z","load_balancing_strategy":"primary","db_duration_s":0.001687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2091vm-test-run-nixbot-gitlab> gitlab # [ 219.948364] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe537c3524ce2f59571154b7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe537c3524ce2f59571154b7: start","job_status":"start","queue_duration_s":1.051947,"scheduling_latency_s":1.051968}2092vm-test-run-nixbot-gitlab> gitlab # [ 219.968005] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe537c3524ce2f59571154b7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe537c3524ce2f59571154b7: done: 0.016086 sec","job_status":"done","queue_duration_s":1.051947,"scheduling_latency_s":1.051968,"redis_calls":13,"redis_duration_s":0.003045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016086,"completed_at":"2026-06-29T21:57:11.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2093vm-test-run-nixbot-gitlab> gitlab # [ 220.055295] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15ab3889499a84cbbc3765ca","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15ab3889499a84cbbc3765ca: start","job_status":"start","queue_duration_s":1.069767,"scheduling_latency_s":1.069787}2094vm-test-run-nixbot-gitlab> gitlab # [ 220.056428] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15ab3889499a84cbbc3765ca","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15ab3889499a84cbbc3765ca: done: 0.015716 sec","job_status":"done","queue_duration_s":1.069767,"scheduling_latency_s":1.069787,"redis_calls":13,"redis_duration_s":0.003048,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015716,"completed_at":"2026-06-29T21:57:11.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2095vm-test-run-nixbot-gitlab> gitlab # [ 220.110334] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63c1eda2fc6cbf48bdb81047","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63c1eda2fc6cbf48bdb81047: start","job_status":"start","queue_duration_s":1.086843,"scheduling_latency_s":1.086864}2096vm-test-run-nixbot-gitlab> gitlab # [ 220.179568] gitaly[1387]: time="2026-06-29T21:57:12.700Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5VANS6Z9T0NPA5QX39JP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:42.698" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:12.699" grpc.time_ms=0.571 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097vm-test-run-nixbot-gitlab> gitlab # [ 220.181092] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 542098vm-test-run-nixbot-gitlab> gitlab # [ 220.181916] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63c1eda2fc6cbf48bdb81047","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63c1eda2fc6cbf48bdb81047: done: 0.015989 sec","job_status":"done","queue_duration_s":1.086843,"scheduling_latency_s":1.086864,"redis_calls":13,"redis_duration_s":0.002982,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015989,"completed_at":"2026-06-29T21:57:11.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2099vm-test-run-nixbot-gitlab> gitlab # [ 220.214765] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f0342d0a6104efad153be12","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f0342d0a6104efad153be12: start","job_status":"start","queue_duration_s":1.10415,"scheduling_latency_s":1.104171}2100vm-test-run-nixbot-gitlab> gitlab # [ 220.234893] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f0342d0a6104efad153be12","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f0342d0a6104efad153be12: done: 0.015751 sec","job_status":"done","queue_duration_s":1.10415,"scheduling_latency_s":1.104171,"redis_calls":13,"redis_duration_s":0.003321,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001015,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002306,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015751,"completed_at":"2026-06-29T21:57:11.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2101vm-test-run-nixbot-gitlab> gitlab # [ 220.340913] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b25babf53dab940c0e3df16","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b25babf53dab940c0e3df16: start","job_status":"start","queue_duration_s":1.121206,"scheduling_latency_s":1.121228}2102vm-test-run-nixbot-gitlab> gitlab # [ 220.342262] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b25babf53dab940c0e3df16","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b25babf53dab940c0e3df16: done: 0.018781 sec","job_status":"done","queue_duration_s":1.121206,"scheduling_latency_s":1.121228,"redis_calls":13,"redis_duration_s":0.003304,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002412,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018781,"completed_at":"2026-06-29T21:57:11.504Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2103vm-test-run-nixbot-gitlab> gitlab # [ 220.379190] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6ad61a4f39dbd14ec60a32c","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6ad61a4f39dbd14ec60a32c: start","job_status":"start","queue_duration_s":1.141312,"scheduling_latency_s":1.141334}2104vm-test-run-nixbot-gitlab> gitlab # [ 220.399108] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6ad61a4f39dbd14ec60a32c","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6ad61a4f39dbd14ec60a32c: done: 0.015753 sec","job_status":"done","queue_duration_s":1.141312,"scheduling_latency_s":1.141334,"redis_calls":13,"redis_duration_s":0.003163,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015753,"completed_at":"2026-06-29T21:57:11.521Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2105vm-test-run-nixbot-gitlab> gitlab # [ 220.487400] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9825288efe5ca08673fb9e9","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9825288efe5ca08673fb9e9: start","job_status":"start","queue_duration_s":1.158763,"scheduling_latency_s":1.158783}2106vm-test-run-nixbot-gitlab> gitlab # [ 220.488949] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9825288efe5ca08673fb9e9","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9825288efe5ca08673fb9e9: done: 0.0161 sec","job_status":"done","queue_duration_s":1.158763,"scheduling_latency_s":1.158783,"redis_calls":13,"redis_duration_s":0.0034460000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002369,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0161,"completed_at":"2026-06-29T21:57:11.539Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2107vm-test-run-nixbot-gitlab> gitlab # [ 220.544916] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c597d82acbad773f4b2c64d0","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c597d82acbad773f4b2c64d0: start","job_status":"start","queue_duration_s":1.176947,"scheduling_latency_s":1.176991}2108vm-test-run-nixbot-gitlab> gitlab # [ 220.620125] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c597d82acbad773f4b2c64d0","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c597d82acbad773f4b2c64d0: done: 0.020729 sec","job_status":"done","queue_duration_s":1.176947,"scheduling_latency_s":1.176991,"redis_calls":13,"redis_duration_s":0.004657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003764,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020729,"completed_at":"2026-06-29T21:57:11.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2109vm-test-run-nixbot-gitlab> gitlab # [ 220.627296] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18d2b3b49c8b9eff60eeb55d","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18d2b3b49c8b9eff60eeb55d: start","job_status":"start","queue_duration_s":1.198958,"scheduling_latency_s":1.19898}2110vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses2111vm-test-run-nixbot-gitlab> gitlab # [ 220.707951] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18d2b3b49c8b9eff60eeb55d","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18d2b3b49c8b9eff60eeb55d: done: 0.017631 sec","job_status":"done","queue_duration_s":1.198958,"scheduling_latency_s":1.19898,"redis_calls":13,"redis_duration_s":0.0036689999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001004,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002665,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017631,"completed_at":"2026-06-29T21:57:11.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2112vm-test-run-nixbot-gitlab> gitlab # [ 220.710784] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e14a9d1bb0a490db35b71e4","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e14a9d1bb0a490db35b71e4: start","job_status":"start","queue_duration_s":1.218518,"scheduling_latency_s":1.218545}2113vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 0.10 seconds)2114vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T21:57:05.776Z', 'started_at': None, 'finished_at': '2026-06-29T21:57:13.494Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T21:57:13.627Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2115vm-test-run-nixbot-gitlab> gitlab # [ 220.731064] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e14a9d1bb0a490db35b71e4","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e14a9d1bb0a490db35b71e4: done: 0.01752 sec","job_status":"done","queue_duration_s":1.218518,"scheduling_latency_s":1.218545,"redis_calls":13,"redis_duration_s":0.003635,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001255,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00238,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01752,"completed_at":"2026-06-29T21:57:11.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2116vm-test-run-nixbot-gitlab> gitlab # [ 220.811475] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"680ea2c8eb33cd3e287c0e34","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-680ea2c8eb33cd3e287c0e34: start","job_status":"start","queue_duration_s":1.237384,"scheduling_latency_s":1.237408}2117vm-test-run-nixbot-gitlab> gitlab # [ 220.812990] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"680ea2c8eb33cd3e287c0e34","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-680ea2c8eb33cd3e287c0e34: done: 0.01542 sec","job_status":"done","queue_duration_s":1.237384,"scheduling_latency_s":1.237408,"redis_calls":13,"redis_duration_s":0.002835,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01542,"completed_at":"2026-06-29T21:57:11.617Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118vm-test-run-nixbot-gitlab> gitlab # [ 220.926416] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb9cb9e4a9d2b2f80d6182b9","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb9cb9e4a9d2b2f80d6182b9: start","job_status":"start","queue_duration_s":1.254316,"scheduling_latency_s":1.254337}2119vm-test-run-nixbot-gitlab> gitlab # [ 220.927950] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb9cb9e4a9d2b2f80d6182b9","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb9cb9e4a9d2b2f80d6182b9: done: 0.015114 sec","job_status":"done","queue_duration_s":1.254316,"scheduling_latency_s":1.254337,"redis_calls":13,"redis_duration_s":0.002826,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015114,"completed_at":"2026-06-29T21:57:11.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2120vm-test-run-nixbot-gitlab> gitlab # [ 221.015231] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"878a7a4846d2fa4379403b01","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-878a7a4846d2fa4379403b01: start","job_status":"start","queue_duration_s":1.271324,"scheduling_latency_s":1.271365}2121vm-test-run-nixbot-gitlab> gitlab # [ 221.016896] gitaly[1387]: time="2026-06-29T21:57:13.472Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:23.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:13.464" grpc.time_ms=7.441 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2122vm-test-run-nixbot-gitlab> gitlab # [ 221.018558] nixbot[3185]: {"ts": "2026-06-29T21:57:13.520Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8b5212701fe6acf76e74f205791f1e298c878a79 \"HTTP/1.1 201 Created\""}2123vm-test-run-nixbot-gitlab> gitlab # [ 221.019328] nixbot[3185]: {"ts": "2026-06-29T21:57:13.647Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8b5212701fe6acf76e74f205791f1e298c878a79 \"HTTP/1.1 201 Created\""}2124vm-test-run-nixbot-gitlab> gitlab # [ 221.020235] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"878a7a4846d2fa4379403b01","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-878a7a4846d2fa4379403b01: done: 0.019745 sec","job_status":"done","queue_duration_s":1.271324,"scheduling_latency_s":1.271365,"redis_calls":13,"redis_duration_s":0.0032770000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002239,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019745,"completed_at":"2026-06-29T21:57:11.655Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125vm-test-run-nixbot-gitlab> gitlab # [ 221.071635] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3d45083deb07fc8aaab3352","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3d45083deb07fc8aaab3352: start","job_status":"start","queue_duration_s":1.292313,"scheduling_latency_s":1.292338}2126vm-test-run-nixbot-gitlab> gitlab # [ 221.091207] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3d45083deb07fc8aaab3352","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3d45083deb07fc8aaab3352: done: 0.024629 sec","job_status":"done","queue_duration_s":1.292313,"scheduling_latency_s":1.292338,"redis_calls":21,"redis_duration_s":0.004542,"redis_read_bytes":18,"redis_write_bytes":2297,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001731,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1140,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024629,"completed_at":"2026-06-29T21:57:11.681Z","load_balancing_strategy":"primary","db_duration_s":0.001585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2127vm-test-run-nixbot-gitlab> gitlab # [ 221.216960] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17f353cb8e2d9f2038d569f6","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17f353cb8e2d9f2038d569f6: start","job_status":"start","queue_duration_s":1.318545,"scheduling_latency_s":1.318568}2128vm-test-run-nixbot-gitlab> gitlab # [ 221.218223] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17f353cb8e2d9f2038d569f6","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17f353cb8e2d9f2038d569f6: done: 0.014999 sec","job_status":"done","queue_duration_s":1.318545,"scheduling_latency_s":1.318568,"redis_calls":13,"redis_duration_s":0.002895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014999,"completed_at":"2026-06-29T21:57:11.698Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2129vm-test-run-nixbot-gitlab> gitlab # [ 221.306210] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"469d9d9536bc941618d41d5f","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-469d9d9536bc941618d41d5f: start","job_status":"start","queue_duration_s":1.334857,"scheduling_latency_s":1.334878}2130vm-test-run-nixbot-gitlab> gitlab # [ 221.307763] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"469d9d9536bc941618d41d5f","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-469d9d9536bc941618d41d5f: done: 0.015019 sec","job_status":"done","queue_duration_s":1.334857,"scheduling_latency_s":1.334878,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015019,"completed_at":"2026-06-29T21:57:11.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2131vm-test-run-nixbot-gitlab> gitlab # [ 221.321608] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d38bc0a4afb1111f35e641f7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d38bc0a4afb1111f35e641f7: start","job_status":"start","queue_duration_s":1.351134,"scheduling_latency_s":1.351154}2132vm-test-run-nixbot-gitlab> gitlab # [ 221.342124] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d38bc0a4afb1111f35e641f7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d38bc0a4afb1111f35e641f7: done: 0.014902 sec","job_status":"done","queue_duration_s":1.351134,"scheduling_latency_s":1.351154,"redis_calls":13,"redis_duration_s":0.0028469999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014902,"completed_at":"2026-06-29T21:57:11.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133vm-test-run-nixbot-gitlab> gitlab # [ 221.407542] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b098fdd6233f7cf02445e38","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b098fdd6233f7cf02445e38: start","job_status":"start","queue_duration_s":1.367461,"scheduling_latency_s":1.367483}2134vm-test-run-nixbot-gitlab> gitlab # [ 221.427843] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b098fdd6233f7cf02445e38","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b098fdd6233f7cf02445e38: done: 0.014913 sec","job_status":"done","queue_duration_s":1.367461,"scheduling_latency_s":1.367483,"redis_calls":13,"redis_duration_s":0.002784,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014913,"completed_at":"2026-06-29T21:57:11.746Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135vm-test-run-nixbot-gitlab> gitlab # [ 221.512311] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f41ff9ee96c51d283c5bedff","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f41ff9ee96c51d283c5bedff: start","job_status":"start","queue_duration_s":1.383702,"scheduling_latency_s":1.383734}2136vm-test-run-nixbot-gitlab> gitlab # [ 221.513891] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f41ff9ee96c51d283c5bedff","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f41ff9ee96c51d283c5bedff: done: 0.014904 sec","job_status":"done","queue_duration_s":1.383702,"scheduling_latency_s":1.383734,"redis_calls":13,"redis_duration_s":0.0028250000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014904,"completed_at":"2026-06-29T21:57:11.763Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2137vm-test-run-nixbot-gitlab> gitlab # [ 221.574964] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b70626c710ef1b4a7b4c82a7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b70626c710ef1b4a7b4c82a7: start","job_status":"start","queue_duration_s":1.400042,"scheduling_latency_s":1.400063}2138vm-test-run-nixbot-gitlab> gitlab # [ 221.657114] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b70626c710ef1b4a7b4c82a7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b70626c710ef1b4a7b4c82a7: done: 0.014976 sec","job_status":"done","queue_duration_s":1.400042,"scheduling_latency_s":1.400063,"redis_calls":13,"redis_duration_s":0.002889,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014976,"completed_at":"2026-06-29T21:57:11.779Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139vm-test-run-nixbot-gitlab> gitlab # [ 221.658721] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22f381c50f2cfe28e30cea6b","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22f381c50f2cfe28e30cea6b: start","job_status":"start","queue_duration_s":1.416311,"scheduling_latency_s":1.416332}2140vm-test-run-nixbot-gitlab> gitlab # [ 221.676056] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22f381c50f2cfe28e30cea6b","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22f381c50f2cfe28e30cea6b: done: 0.015208 sec","job_status":"done","queue_duration_s":1.416311,"scheduling_latency_s":1.416332,"redis_calls":13,"redis_duration_s":0.002873,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015208,"completed_at":"2026-06-29T21:57:11.796Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses2142vm-test-run-nixbot-gitlab> gitlab # [ 221.782198] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4cd46c1a0241dc69b9dcb20","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4cd46c1a0241dc69b9dcb20: start","job_status":"start","queue_duration_s":1.432871,"scheduling_latency_s":1.432893}2143vm-test-run-nixbot-gitlab> gitlab # [ 221.783473] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4cd46c1a0241dc69b9dcb20","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4cd46c1a0241dc69b9dcb20: done: 0.014957 sec","job_status":"done","queue_duration_s":1.432871,"scheduling_latency_s":1.432893,"redis_calls":13,"redis_duration_s":0.002779,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014957,"completed_at":"2026-06-29T21:57:11.812Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2144vm-test-run-nixbot-gitlab> gitlab # [ 221.822362] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c34cdaa3aef1b3b062426a3","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c34cdaa3aef1b3b062426a3: start","job_status":"start","queue_duration_s":1.449093,"scheduling_latency_s":1.449114}2145vm-test-run-nixbot-gitlab> gitlab # [ 221.842955] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8b5212701fe6acf76e74f205791f1e298c878a79 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 23172146vm-test-run-nixbot-gitlab> gitlab # [ 221.846289] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8b5212701fe6acf76e74f205791f1e298c878a79 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1222147vm-test-run-nixbot-gitlab> gitlab # [ 221.850323] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 602148vm-test-run-nixbot-gitlab> gitlab # [ 221.855303] gitaly[1387]: time="2026-06-29T21:57:13.569Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5W5KJK600217XE3NET19 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:43.568" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:13.569" grpc.time_ms=0.646 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2149vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 0.15 seconds)2150vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T21:57:05.776Z', 'started_at': None, 'finished_at': '2026-06-29T21:57:13.494Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T21:57:13.627Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2151vm-test-run-nixbot-gitlab> gitlab # [ 221.875606] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c34cdaa3aef1b3b062426a3","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c34cdaa3aef1b3b062426a3: done: 0.014959 sec","job_status":"done","queue_duration_s":1.449093,"scheduling_latency_s":1.449114,"redis_calls":13,"redis_duration_s":0.0028499999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014959,"completed_at":"2026-06-29T21:57:11.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2152vm-test-run-nixbot-gitlab> gitlab # [ 221.940262] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5463ad90787f9c585eccda6d","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5463ad90787f9c585eccda6d: start","job_status":"start","queue_duration_s":1.465497,"scheduling_latency_s":1.465519}2153vm-test-run-nixbot-gitlab> gitlab # [ 221.960494] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5463ad90787f9c585eccda6d","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5463ad90787f9c585eccda6d: done: 0.015237 sec","job_status":"done","queue_duration_s":1.465497,"scheduling_latency_s":1.465519,"redis_calls":13,"redis_duration_s":0.0029349999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015237,"completed_at":"2026-06-29T21:57:11.845Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154vm-test-run-nixbot-gitlab> gitlab # [ 222.040093] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a794835e1d5ff8b2e729e915","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a794835e1d5ff8b2e729e915: start","job_status":"start","queue_duration_s":1.482028,"scheduling_latency_s":1.482049}2155vm-test-run-nixbot-gitlab> gitlab # [ 222.042269] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a794835e1d5ff8b2e729e915","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a794835e1d5ff8b2e729e915: done: 0.014875 sec","job_status":"done","queue_duration_s":1.482028,"scheduling_latency_s":1.482049,"redis_calls":13,"redis_duration_s":0.002884,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014875,"completed_at":"2026-06-29T21:57:11.861Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156vm-test-run-nixbot-gitlab> gitlab # [ 222.106639] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d793e72becc413aa5ab6077","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d793e72becc413aa5ab6077: start","job_status":"start","queue_duration_s":1.49818,"scheduling_latency_s":1.4982}2157vm-test-run-nixbot-gitlab> gitlab # [ 222.126360] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d793e72becc413aa5ab6077","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d793e72becc413aa5ab6077: done: 0.015101 sec","job_status":"done","queue_duration_s":1.49818,"scheduling_latency_s":1.4982,"redis_calls":13,"redis_duration_s":0.002899,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015101,"completed_at":"2026-06-29T21:57:11.877Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2158vm-test-run-nixbot-gitlab> gitlab # [ 222.243563] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d822d8c331ec1a4ae6fc95c","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d822d8c331ec1a4ae6fc95c: start","job_status":"start","queue_duration_s":1.514684,"scheduling_latency_s":1.514705}2159vm-test-run-nixbot-gitlab> gitlab # [ 222.245083] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d822d8c331ec1a4ae6fc95c","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d822d8c331ec1a4ae6fc95c: done: 0.015044 sec","job_status":"done","queue_duration_s":1.514684,"scheduling_latency_s":1.514705,"redis_calls":13,"redis_duration_s":0.0029509999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015044,"completed_at":"2026-06-29T21:57:11.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160vm-test-run-nixbot-gitlab> gitlab # [ 222.270457] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bce1c7e1f820d9f1e8733d7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bce1c7e1f820d9f1e8733d7: start","job_status":"start","queue_duration_s":1.53112,"scheduling_latency_s":1.531141}2161vm-test-run-nixbot-gitlab> gitlab # [ 222.290897] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bce1c7e1f820d9f1e8733d7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bce1c7e1f820d9f1e8733d7: done: 0.014918 sec","job_status":"done","queue_duration_s":1.53112,"scheduling_latency_s":1.531141,"redis_calls":13,"redis_duration_s":0.00285,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014918,"completed_at":"2026-06-29T21:57:11.910Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2162vm-test-run-nixbot-gitlab> gitlab # [ 222.390302] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.911Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"591c9be8c1387391252b477a","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-591c9be8c1387391252b477a: start","job_status":"start","queue_duration_s":1.547306,"scheduling_latency_s":1.547328}2163vm-test-run-nixbot-gitlab> gitlab # [ 222.391847] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"591c9be8c1387391252b477a","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-591c9be8c1387391252b477a: done: 0.015275 sec","job_status":"done","queue_duration_s":1.547306,"scheduling_latency_s":1.547328,"redis_calls":13,"redis_duration_s":0.002954,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015275,"completed_at":"2026-06-29T21:57:11.927Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164vm-test-run-nixbot-gitlab> gitlab # [ 222.436404] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"deffe9177e8cdb6e888d529b","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-deffe9177e8cdb6e888d529b: start","job_status":"start","queue_duration_s":1.56389,"scheduling_latency_s":1.563911}2165vm-test-run-nixbot-gitlab> gitlab # [ 222.456490] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"deffe9177e8cdb6e888d529b","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-deffe9177e8cdb6e888d529b: done: 0.01484 sec","job_status":"done","queue_duration_s":1.56389,"scheduling_latency_s":1.563911,"redis_calls":13,"redis_duration_s":0.002795,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01484,"completed_at":"2026-06-29T21:57:11.943Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166vm-test-run-nixbot-gitlab> gitlab # [ 222.537063] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4642003c4315f33100385cb5","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4642003c4315f33100385cb5: start","job_status":"start","queue_duration_s":1.580008,"scheduling_latency_s":1.580028}2167vm-test-run-nixbot-gitlab> gitlab # [ 222.538081] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4642003c4315f33100385cb5","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4642003c4315f33100385cb5: done: 0.015087 sec","job_status":"done","queue_duration_s":1.580008,"scheduling_latency_s":1.580028,"redis_calls":13,"redis_duration_s":0.002964,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015087,"completed_at":"2026-06-29T21:57:11.959Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168vm-test-run-nixbot-gitlab> gitlab # [ 222.602452] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd302c28cb519bef1233c03d","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd302c28cb519bef1233c03d: start","job_status":"start","queue_duration_s":1.596375,"scheduling_latency_s":1.596397}2169vm-test-run-nixbot-gitlab> gitlab # [ 222.622989] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd302c28cb519bef1233c03d","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd302c28cb519bef1233c03d: done: 0.014982 sec","job_status":"done","queue_duration_s":1.596375,"scheduling_latency_s":1.596397,"redis_calls":13,"redis_duration_s":0.00285,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014982,"completed_at":"2026-06-29T21:57:11.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2170vm-test-run-nixbot-gitlab> gitlab # [ 222.733978] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b8bf672058783084267995c","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b8bf672058783084267995c: start","job_status":"start","queue_duration_s":1.612787,"scheduling_latency_s":1.612808}2171vm-test-run-nixbot-gitlab> gitlab # [ 222.735557] gitaly[1387]: time="2026-06-29T21:57:13.604Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5W5KJK600217XE3NET19 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:23.186" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:13.597" grpc.time_ms=6.895 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2172vm-test-run-nixbot-gitlab> gitlab # [ 222.736818] gitaly[1387]: time="2026-06-29T21:57:13.614Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:13.613" grpc.time_ms=0.352 pid=1387 span.kind=server system=grpc2173vm-test-run-nixbot-gitlab> gitlab # [ 222.742486] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 882174vm-test-run-nixbot-gitlab> gitlab # [ 222.747160] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b8bf672058783084267995c","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b8bf672058783084267995c: done: 0.014886 sec","job_status":"done","queue_duration_s":1.612787,"scheduling_latency_s":1.612808,"redis_calls":13,"redis_duration_s":0.0027860000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014886,"completed_at":"2026-06-29T21:57:11.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2175vm-test-run-nixbot-gitlab> gitlab # [ 222.830109] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:11.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"559f3aea4bc5c0db74ce7403","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-559f3aea4bc5c0db74ce7403: start","job_status":"start","queue_duration_s":1.629441,"scheduling_latency_s":1.629486}2176vm-test-run-nixbot-gitlab> gitlab # [ 222.831512] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"559f3aea4bc5c0db74ce7403","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-559f3aea4bc5c0db74ce7403: done: 0.021872 sec","job_status":"done","queue_duration_s":1.629441,"scheduling_latency_s":1.629486,"redis_calls":15,"redis_duration_s":0.003712,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002327,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021872,"completed_at":"2026-06-29T21:57:12.015Z","load_balancing_strategy":"primary","db_duration_s":0.000615,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2177vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses2178vm-test-run-nixbot-gitlab> gitlab # [ 222.895981] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bb51550aa5cebf49e1ee79e","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bb51550aa5cebf49e1ee79e: start","job_status":"start","queue_duration_s":1.652566,"scheduling_latency_s":1.652632}2179vm-test-run-nixbot-gitlab> gitlab # [ 222.975472] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bb51550aa5cebf49e1ee79e","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bb51550aa5cebf49e1ee79e: done: 0.015166 sec","job_status":"done","queue_duration_s":1.652566,"scheduling_latency_s":1.652632,"redis_calls":13,"redis_duration_s":0.00281,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015166,"completed_at":"2026-06-29T21:57:12.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180vm-test-run-nixbot-gitlab> gitlab # [ 222.977191] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e35a17ff8a8407084d11865","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e35a17ff8a8407084d11865: start","job_status":"start","queue_duration_s":1.669134,"scheduling_latency_s":1.669155}2181vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 0.14 seconds)2182vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T21:57:05.776Z', 'started_at': None, 'finished_at': '2026-06-29T21:57:13.494Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T21:57:13.627Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2183vm-test-run-nixbot-gitlab> gitlab # [ 222.997406] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e35a17ff8a8407084d11865","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e35a17ff8a8407084d11865: done: 0.015011 sec","job_status":"done","queue_duration_s":1.669134,"scheduling_latency_s":1.669155,"redis_calls":13,"redis_duration_s":0.002868,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015011,"completed_at":"2026-06-29T21:57:12.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184vm-test-run-nixbot-gitlab> gitlab # [ 223.122816] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40e7c14eb0bb2a5d41964c22","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40e7c14eb0bb2a5d41964c22: start","job_status":"start","queue_duration_s":1.685529,"scheduling_latency_s":1.68555}2185vm-test-run-nixbot-gitlab> gitlab # [ 223.124300] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40e7c14eb0bb2a5d41964c22","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40e7c14eb0bb2a5d41964c22: done: 0.01504 sec","job_status":"done","queue_duration_s":1.685529,"scheduling_latency_s":1.68555,"redis_calls":13,"redis_duration_s":0.002845,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01504,"completed_at":"2026-06-29T21:57:12.065Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2186vm-test-run-nixbot-gitlab> gitlab # [ 223.143987] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c66ca3a4e4cf830444c242b","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c66ca3a4e4cf830444c242b: start","job_status":"start","queue_duration_s":1.701886,"scheduling_latency_s":1.701908}2187vm-test-run-nixbot-gitlab> gitlab # [ 223.212619] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c66ca3a4e4cf830444c242b","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c66ca3a4e4cf830444c242b: done: 0.015859 sec","job_status":"done","queue_duration_s":1.701886,"scheduling_latency_s":1.701908,"redis_calls":13,"redis_duration_s":0.0031130000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015859,"completed_at":"2026-06-29T21:57:12.082Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2188vm-test-run-nixbot-gitlab> gitlab # [ 223.225971] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f828176f38d65e5a10f3936","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f828176f38d65e5a10f3936: start","job_status":"start","queue_duration_s":1.719049,"scheduling_latency_s":1.719071}2189vm-test-run-nixbot-gitlab> gitlab # [ 223.308926] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f828176f38d65e5a10f3936","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f828176f38d65e5a10f3936: done: 0.015423 sec","job_status":"done","queue_duration_s":1.719049,"scheduling_latency_s":1.719071,"redis_calls":13,"redis_duration_s":0.003054,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015423,"completed_at":"2026-06-29T21:57:12.099Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2190vm-test-run-nixbot-gitlab> gitlab # [ 223.311154] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97041033e4be2a5ada2a69c7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97041033e4be2a5ada2a69c7: start","job_status":"start","queue_duration_s":1.736011,"scheduling_latency_s":1.736032}2191vm-test-run-nixbot-gitlab> gitlab # [ 223.328308] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97041033e4be2a5ada2a69c7","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97041033e4be2a5ada2a69c7: done: 0.015159 sec","job_status":"done","queue_duration_s":1.736011,"scheduling_latency_s":1.736032,"redis_calls":13,"redis_duration_s":0.00277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015159,"completed_at":"2026-06-29T21:57:12.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2192vm-test-run-nixbot-gitlab> gitlab # [ 223.409293] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6c6cf29ef92de4d0eae41b2","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6c6cf29ef92de4d0eae41b2: start","job_status":"start","queue_duration_s":1.752443,"scheduling_latency_s":1.752464}2193vm-test-run-nixbot-gitlab> gitlab # [ 223.410817] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6c6cf29ef92de4d0eae41b2","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6c6cf29ef92de4d0eae41b2: done: 0.015128 sec","job_status":"done","queue_duration_s":1.752443,"scheduling_latency_s":1.752464,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015128,"completed_at":"2026-06-29T21:57:12.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2194vm-test-run-nixbot-gitlab> gitlab # [ 223.473954] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dc6c30600e46fce69880d62","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dc6c30600e46fce69880d62: start","job_status":"start","queue_duration_s":1.768899,"scheduling_latency_s":1.768921}2195vm-test-run-nixbot-gitlab> gitlab # [ 223.558604] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dc6c30600e46fce69880d62","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dc6c30600e46fce69880d62: done: 0.015114 sec","job_status":"done","queue_duration_s":1.768899,"scheduling_latency_s":1.768921,"redis_calls":13,"redis_duration_s":0.00283,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015114,"completed_at":"2026-06-29T21:57:12.148Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2196vm-test-run-nixbot-gitlab> gitlab # [ 223.560733] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95e462c59832c62fc5794847","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95e462c59832c62fc5794847: start","job_status":"start","queue_duration_s":1.785478,"scheduling_latency_s":1.785499}2197vm-test-run-nixbot-gitlab> gitlab # [ 223.576664] gitaly[1387]: time="2026-06-29T21:57:13.641Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:13.640" grpc.time_ms=0.647 pid=1387 span.kind=server system=grpc2198vm-test-run-nixbot-gitlab> gitlab # [ 223.591438] gitaly[1387]: time="2026-06-29T21:57:13.667Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:13.666" grpc.time_ms=0.615 pid=1387 span.kind=server system=grpc2199vm-test-run-nixbot-gitlab> gitlab # [ 223.607518] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 842200vm-test-run-nixbot-gitlab> gitlab # [ 223.611490] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95e462c59832c62fc5794847","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95e462c59832c62fc5794847: done: 0.015249 sec","job_status":"done","queue_duration_s":1.785478,"scheduling_latency_s":1.785499,"redis_calls":13,"redis_duration_s":0.002931,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015249,"completed_at":"2026-06-29T21:57:12.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201vm-test-run-nixbot-gitlab> gitlab # [ 223.705520] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c64f11d80293af5e7de1fece","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c64f11d80293af5e7de1fece: start","job_status":"start","queue_duration_s":1.802049,"scheduling_latency_s":1.802072}2202vm-test-run-nixbot-gitlab> gitlab # [ 223.706722] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c64f11d80293af5e7de1fece","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c64f11d80293af5e7de1fece: done: 0.015256 sec","job_status":"done","queue_duration_s":1.802049,"scheduling_latency_s":1.802072,"redis_calls":13,"redis_duration_s":0.0029200000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015256,"completed_at":"2026-06-29T21:57:12.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203vm-test-run-nixbot-gitlab> gitlab # [ 223.758227] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c67360210f1e56b6df36ceb1","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c67360210f1e56b6df36ceb1: start","job_status":"start","queue_duration_s":1.818637,"scheduling_latency_s":1.818657}2204vm-test-run-nixbot-gitlab> gitlab # [ 223.836743] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c67360210f1e56b6df36ceb1","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c67360210f1e56b6df36ceb1: done: 0.015185 sec","job_status":"done","queue_duration_s":1.818637,"scheduling_latency_s":1.818657,"redis_calls":13,"redis_duration_s":0.002818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015185,"completed_at":"2026-06-29T21:57:12.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2205vm-test-run-nixbot-gitlab> gitlab # [ 223.840181] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd9815919e31e92be7d280c8","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd9815919e31e92be7d280c8: start","job_status":"start","queue_duration_s":1.835047,"scheduling_latency_s":1.835067}2206vm-test-run-nixbot-gitlab> gitlab # [ 223.861461] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd9815919e31e92be7d280c8","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd9815919e31e92be7d280c8: done: 0.018319 sec","job_status":"done","queue_duration_s":1.835047,"scheduling_latency_s":1.835067,"redis_calls":13,"redis_duration_s":0.0032059999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018319,"completed_at":"2026-06-29T21:57:12.217Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2207vm-test-run-nixbot-gitlab> gitlab # [ 223.973098] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7bd7f4854bceb1eaf234c2e","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7bd7f4854bceb1eaf234c2e: start","job_status":"start","queue_duration_s":1.854933,"scheduling_latency_s":1.854961}2208vm-test-run-nixbot-gitlab> gitlab # [ 223.975089] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7bd7f4854bceb1eaf234c2e","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7bd7f4854bceb1eaf234c2e: done: 0.015699 sec","job_status":"done","queue_duration_s":1.854933,"scheduling_latency_s":1.854961,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015699,"completed_at":"2026-06-29T21:57:12.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2209vm-test-run-nixbot-gitlab> gitlab # [ 224.007975] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19d77faaec89941bf8b00c4b","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19d77faaec89941bf8b00c4b: start","job_status":"start","queue_duration_s":1.872096,"scheduling_latency_s":1.872118}2210vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses2211vm-test-run-nixbot-gitlab> gitlab # [ 224.027776] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19d77faaec89941bf8b00c4b","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19d77faaec89941bf8b00c4b: done: 0.015045 sec","job_status":"done","queue_duration_s":1.872096,"scheduling_latency_s":1.872118,"redis_calls":13,"redis_duration_s":0.002858,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015045,"completed_at":"2026-06-29T21:57:12.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212vm-test-run-nixbot-gitlab> gitlab # [ 224.147591] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96c79715ba857082ae930681","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96c79715ba857082ae930681: start","job_status":"start","queue_duration_s":1.888999,"scheduling_latency_s":1.889035}2213vm-test-run-nixbot-gitlab> gitlab # [ 224.148752] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96c79715ba857082ae930681","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96c79715ba857082ae930681: done: 0.017464 sec","job_status":"done","queue_duration_s":1.888999,"scheduling_latency_s":1.889035,"redis_calls":13,"redis_duration_s":0.0028870000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017464,"completed_at":"2026-06-29T21:57:12.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2214vm-test-run-nixbot-gitlab> gitlab # [ 224.170728] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc692edf7a1480ca5a2397e2","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc692edf7a1480ca5a2397e2: start","job_status":"start","queue_duration_s":1.907784,"scheduling_latency_s":1.907804}2215vm-test-run-nixbot-gitlab> gitlab # [ 224.190308] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc692edf7a1480ca5a2397e2","created_at":"2026-06-29T21:57:10.298Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-315f5fef6e8c41e1","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d44af36127b16d5f17286e642a1f11ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.364Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc692edf7a1480ca5a2397e2: done: 0.015051 sec","job_status":"done","queue_duration_s":1.907784,"scheduling_latency_s":1.907804,"redis_calls":13,"redis_duration_s":0.002857,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015051,"completed_at":"2026-06-29T21:57:12.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216vm-test-run-nixbot-gitlab> gitlab # [ 224.253206] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.291Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"aeb4be3af2d569a6f6f9fa82","created_at":"2026-06-29T21:57:10.430Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-a343ccb973f84c37","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.436Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-aeb4be3af2d569a6f6f9fa82: start","job_status":"start","queue_duration_s":1.854467,"scheduling_latency_s":1.85449}2217vm-test-run-nixbot-gitlab> gitlab # [ 224.272935] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.304Z","message":"No existing merge request to be cleaned up."}2218vm-test-run-nixbot-gitlab> gitlab # [ 224.275763] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.313Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"aeb4be3af2d569a6f6f9fa82","created_at":"2026-06-29T21:57:10.430Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-a343ccb973f84c37","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.436Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-aeb4be3af2d569a6f6f9fa82: done: 0.022256 sec","job_status":"done","queue_duration_s":1.854467,"scheduling_latency_s":1.85449,"redis_calls":12,"redis_duration_s":0.003585,"redis_read_bytes":10,"redis_write_bytes":1377,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000487,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":355,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001101,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022256,"completed_at":"2026-06-29T21:57:12.313Z","load_balancing_strategy":"primary","db_duration_s":0.002361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2219vm-test-run-nixbot-gitlab> gitlab # [ 224.360266] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.314Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"79b5e08f249ac8e1e3950155","created_at":"2026-06-29T21:57:10.430Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-a343ccb973f84c37","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.436Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-79b5e08f249ac8e1e3950155: start","job_status":"start","queue_duration_s":1.878044,"scheduling_latency_s":1.878067}2220vm-test-run-nixbot-gitlab> gitlab # [ 224.361734] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.324Z","message":"No existing merge request to be cleaned up."}2221vm-test-run-nixbot-gitlab> gitlab # [ 224.362891] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.330Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"79b5e08f249ac8e1e3950155","created_at":"2026-06-29T21:57:10.430Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-a343ccb973f84c37","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.436Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-79b5e08f249ac8e1e3950155: done: 0.016133 sec","job_status":"done","queue_duration_s":1.878044,"scheduling_latency_s":1.878067,"redis_calls":10,"redis_duration_s":0.002244,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001345,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016133,"completed_at":"2026-06-29T21:57:12.330Z","load_balancing_strategy":"primary","db_duration_s":0.001882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2222vm-test-run-nixbot-gitlab> gitlab # [ 224.422935] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.332Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fa79eb19c730977ed771fcdd","created_at":"2026-06-29T21:57:10.430Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-a343ccb973f84c37","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.436Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fa79eb19c730977ed771fcdd: start","job_status":"start","queue_duration_s":1.895755,"scheduling_latency_s":1.895794}2223vm-test-run-nixbot-gitlab> gitlab # [ 224.505143] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.344Z","message":"No existing merge request to be cleaned up."}2224vm-test-run-nixbot-gitlab> gitlab # [ 224.507315] gitaly[1387]: time="2026-06-29T21:57:13.677Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN diskcache=e03a7915-b9e3-433b-b445-3e84c97270ec grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:43.584" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:13.585" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2225vm-test-run-nixbot-gitlab> gitlab # [ 224.508506] gitaly[1387]: time="2026-06-29T21:57:13.677Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=83 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7065 command.oublock=0 command.real_time_ms=84 command.spawn_token_fork_ms=1 command.system_time_ms=51 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:43.584" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:13.585" grpc.time_ms=92.092 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2226vm-test-run-nixbot-gitlab> gitlab # [ 224.509495] gitlab-workhorse[1305]: gitlab 2001:db8:1::1 - - [2026/06/29:21:57:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8b5212701fe6acf76e74f205791f1e298c878a79 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2592227vm-test-run-nixbot-gitlab> gitlab # [ 224.510482] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.350Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fa79eb19c730977ed771fcdd","created_at":"2026-06-29T21:57:10.430Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-a343ccb973f84c37","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.436Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fa79eb19c730977ed771fcdd: done: 0.018273 sec","job_status":"done","queue_duration_s":1.895755,"scheduling_latency_s":1.895794,"redis_calls":10,"redis_duration_s":0.002405,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001388,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018273,"completed_at":"2026-06-29T21:57:12.350Z","load_balancing_strategy":"primary","db_duration_s":0.00185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2228vm-test-run-nixbot-gitlab> gitlab # [ 224.594079] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.352Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b781f5cf58f938481842e874","created_at":"2026-06-29T21:57:10.430Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-a343ccb973f84c37","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.436Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b781f5cf58f938481842e874: start","job_status":"start","queue_duration_s":1.915503,"scheduling_latency_s":1.915555}2229vm-test-run-nixbot-gitlab> gitlab # [ 224.597352] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.362Z","message":"No existing merge request to be cleaned up."}2230vm-test-run-nixbot-gitlab> gitlab # [ 224.600163] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.369Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b781f5cf58f938481842e874","created_at":"2026-06-29T21:57:10.430Z","trace_propagation_headers":{"sentry-trace":"52d4ac2bc45f4dabb11d54152f65fa8c-a343ccb973f84c37","baggage":"sentry-trace_id=52d4ac2bc45f4dabb11d54152f65fa8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ea6ca1298983b64447ac0bb7ef743598","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T21:57:10.436Z","job_size_bytes":2,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b781f5cf58f938481842e874: done: 0.017495 sec","job_status":"done","queue_duration_s":1.915503,"scheduling_latency_s":1.915555,"redis_calls":10,"redis_duration_s":0.002633,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001759,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017495,"completed_at":"2026-06-29T21:57:12.369Z","load_balancing_strategy":"primary","db_duration_s":0.001712,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231vm-test-run-nixbot-gitlab> gitlab # [ 224.630437] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:12.858Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"50e0f3531998ee0dd815fdd4","created_at":"2026-06-29T21:56:53.675Z","trace_propagation_headers":{"sentry-trace":"f20e15b3f00d4de89c812f378684ce0f-fd7303d43ba44004","baggage":"sentry-trace_id=f20e15b3f00d4de89c812f378684ce0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58DECNK87Q48HZRHSSBW","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:34353274e7fb3dcc67978427edde804cdf16fcc6199a4ee97c68f52208252e31","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:12.854Z","retry_count":0,"job_size_bytes":451,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-50e0f3531998ee0dd815fdd4: start","job_status":"start","queue_duration_s":0.00363,"scheduling_latency_s":0.003663}2232vm-test-run-nixbot-gitlab> gitlab # [ 224.657297] sidekiq-cluster[2173]: {"severity":"WARN","time":"2026-06-29T21:57:12.894Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"50e0f3531998ee0dd815fdd4","created_at":"2026-06-29T21:56:53.675Z","trace_propagation_headers":{"sentry-trace":"f20e15b3f00d4de89c812f378684ce0f-fd7303d43ba44004","baggage":"sentry-trace_id=f20e15b3f00d4de89c812f378684ce0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP58DECNK87Q48HZRHSSBW","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:34353274e7fb3dcc67978427edde804cdf16fcc6199a4ee97c68f52208252e31","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:12.854Z","retry_count":0,"job_size_bytes":451,"pid":2173,"sidekiq_tid":"m971","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-50e0f3531998ee0dd815fdd4: fail: 0.035429 sec","job_status":"fail","queue_duration_s":0.00363,"scheduling_latency_s":0.003663,"redis_calls":7,"redis_duration_s":0.003354,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002356,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035429,"completed_at":"2026-06-29T21:57:12.893Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.007337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2233vm-test-run-nixbot-gitlab> gitlab # [ 225.268825] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.507Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"58eba83b90b7fdba2ae43f37","created_at":"2026-06-29T21:57:13.502Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-d89547c550e7450c","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.505Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-58eba83b90b7fdba2ae43f37: start","job_status":"start","queue_duration_s":0.002106,"scheduling_latency_s":0.002192}2234vm-test-run-nixbot-gitlab> gitlab # [ 225.293237] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.791Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6f143fe7945487abe53a02ca","created_at":"2026-06-29T21:57:13.788Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"58eba83b90b7fdba2ae43f37","job_size_bytes":3,"pid":2173,"job_status":"deduplicated","message":"PipelineProcessWorker JID-6f143fe7945487abe53a02ca: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2235vm-test-run-nixbot-gitlab> gitlab # [ 225.378405] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.793Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"58eba83b90b7fdba2ae43f37","created_at":"2026-06-29T21:57:13.502Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-d89547c550e7450c","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.505Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2173,"job_status":"rescheduled","message":"PipelineProcessWorker JID-58eba83b90b7fdba2ae43f37: rescheduled"}2236vm-test-run-nixbot-gitlab> gitlab # [ 225.380478] nix-daemon[3466]: accepted connection from pid 3688, user nixbot2237vm-test-run-nixbot-gitlab> gitlab # [ 225.381371] nix-daemon[3466]: accepted connection from pid 3734, user nixbot2238vm-test-run-nixbot-gitlab> gitlab # [ 225.382536] nixbot[3185]: {"ts": "2026-06-29T21:57:17.396Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8b5212701fe6acf76e74f205791f1e298c878a79 \"HTTP/1.1 201 Created\""}2239vm-test-run-nixbot-gitlab> gitlab # [ 225.383638] gitaly[1387]: time="2026-06-29T21:57:13.722Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T21:57:43.720" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:13.721" grpc.time_ms=1.171 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2240vm-test-run-nixbot-gitlab> gitlab # [ 225.384793] gitlab-workhorse[1305]: gitlab 127.0.0.1 - - [2026/06/29:21:57:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 612241vm-test-run-nixbot-gitlab> gitlab # [ 225.385803] gitlab-workhorse[1305]: time="2026-06-29T21:57:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672242vm-test-run-nixbot-gitlab> gitlab # [ 225.386699] gitlab-workhorse[1305]: gitlab 127.0.0.1 - - [2026/06/29:21:57:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 2172243vm-test-run-nixbot-gitlab> gitlab # [ 225.387912] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.800Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"58eba83b90b7fdba2ae43f37","created_at":"2026-06-29T21:57:13.502Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-d89547c550e7450c","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.505Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-58eba83b90b7fdba2ae43f37: done: 0.29351 sec","job_status":"done","queue_duration_s":0.002106,"scheduling_latency_s":0.002192,"gitaly_calls":4,"gitaly_duration_s":0.105003,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.020272,"redis_read_bytes":262,"redis_write_bytes":23385,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001076,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002639,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000867,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.003756,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14466,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.007156,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00024,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000617,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002845,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.137185,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.252738826,"duration_s":0.29351,"completed_at":"2026-06-29T21:57:13.800Z","load_balancing_strategy":"primary","db_duration_s":0.037362,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2244vm-test-run-nixbot-gitlab> gitlab # [ 225.481459] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.802Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9526b4054387437218656b36","created_at":"2026-06-29T21:57:13.577Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.579Z","job_size_bytes":23,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9526b4054387437218656b36: start","job_status":"start","queue_duration_s":0.223033,"scheduling_latency_s":0.223061}2245vm-test-run-nixbot-gitlab> gitlab # [ 225.484059] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.842Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9526b4054387437218656b36","created_at":"2026-06-29T21:57:13.577Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.579Z","job_size_bytes":23,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9526b4054387437218656b36: done: 0.039605 sec","job_status":"done","queue_duration_s":0.223033,"scheduling_latency_s":0.223061,"redis_calls":7,"redis_duration_s":0.001791,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039605,"completed_at":"2026-06-29T21:57:13.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007666,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2246vm-test-run-nixbot-gitlab> gitlab # [ 225.552421] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.843Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2c73796bc93d5368dda6eb16","created_at":"2026-06-29T21:57:13.682Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.685Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2c73796bc93d5368dda6eb16: start","job_status":"start","queue_duration_s":0.158318,"scheduling_latency_s":0.158346}2247vm-test-run-nixbot-gitlab> gitlab # [ 225.578771] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.862Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2c73796bc93d5368dda6eb16","created_at":"2026-06-29T21:57:13.682Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.685Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2c73796bc93d5368dda6eb16: done: 0.018992 sec","job_status":"done","queue_duration_s":0.158318,"scheduling_latency_s":0.158346,"redis_calls":7,"redis_duration_s":0.0015890000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018992,"completed_at":"2026-06-29T21:57:13.862Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248vm-test-run-nixbot-gitlab> gitlab # [ 225.664270] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.864Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"40c3cb76ef92bf07f38081a2","created_at":"2026-06-29T21:57:13.692Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.696Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-40c3cb76ef92bf07f38081a2: start","job_status":"start","queue_duration_s":0.167628,"scheduling_latency_s":0.167649}2249vm-test-run-nixbot-gitlab> gitlab # [ 225.669771] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.885Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"40c3cb76ef92bf07f38081a2","created_at":"2026-06-29T21:57:13.692Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.696Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-40c3cb76ef92bf07f38081a2: done: 0.021841 sec","job_status":"done","queue_duration_s":0.167628,"scheduling_latency_s":0.167649,"redis_calls":7,"redis_duration_s":0.00173,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021841,"completed_at":"2026-06-29T21:57:13.885Z","load_balancing_strategy":"primary","db_duration_s":0.004105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2250vm-test-run-nixbot-gitlab> gitlab # [ 225.737533] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.887Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"26a21e474fde359e51a15bbf","created_at":"2026-06-29T21:57:13.703Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.705Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-26a21e474fde359e51a15bbf: start","job_status":"start","queue_duration_s":0.181985,"scheduling_latency_s":0.182006}2251vm-test-run-nixbot-gitlab> gitlab # [ 225.807089] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.908Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"26a21e474fde359e51a15bbf","created_at":"2026-06-29T21:57:13.703Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.705Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-26a21e474fde359e51a15bbf: done: 0.021546 sec","job_status":"done","queue_duration_s":0.181985,"scheduling_latency_s":0.182006,"redis_calls":10,"redis_duration_s":0.0022400000000000002,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001095,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000679,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021546,"completed_at":"2026-06-29T21:57:13.908Z","load_balancing_strategy":"primary","db_duration_s":0.002553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2252vm-test-run-nixbot-gitlab> gitlab # [ 225.829380] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.910Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"416d7c1491c6d603c2f9fd7c","created_at":"2026-06-29T21:57:13.705Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.706Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-416d7c1491c6d603c2f9fd7c: start","job_status":"start","queue_duration_s":0.203123,"scheduling_latency_s":0.203145}2253vm-test-run-nixbot-gitlab> gitlab # [ 225.855504] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.928Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"416d7c1491c6d603c2f9fd7c","created_at":"2026-06-29T21:57:13.705Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.706Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-416d7c1491c6d603c2f9fd7c: done: 0.018157 sec","job_status":"done","queue_duration_s":0.203123,"scheduling_latency_s":0.203145,"redis_calls":7,"redis_duration_s":0.0016460000000000001,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000476,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018157,"completed_at":"2026-06-29T21:57:13.928Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002994,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2254vm-test-run-nixbot-gitlab> gitlab # [ 225.950265] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.930Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"eac03377e0fc94c1741fcece","created_at":"2026-06-29T21:57:13.707Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.709Z","job_size_bytes":19,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-eac03377e0fc94c1741fcece: start","job_status":"start","queue_duration_s":0.22103,"scheduling_latency_s":0.221073}2255vm-test-run-nixbot-gitlab> gitlab # [ 225.951752] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.951Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"eac03377e0fc94c1741fcece","created_at":"2026-06-29T21:57:13.707Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.709Z","job_size_bytes":19,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-eac03377e0fc94c1741fcece: done: 0.021251 sec","job_status":"done","queue_duration_s":0.22103,"scheduling_latency_s":0.221073,"redis_calls":8,"redis_duration_s":0.002007,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00151,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000497,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021251,"completed_at":"2026-06-29T21:57:13.951Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2256vm-test-run-nixbot-gitlab> gitlab # [ 226.014859] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.953Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3969a755c7d9969293208b5c","created_at":"2026-06-29T21:57:13.710Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.711Z","job_size_bytes":66,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3969a755c7d9969293208b5c: start","job_status":"start","queue_duration_s":0.241529,"scheduling_latency_s":0.241554}2257vm-test-run-nixbot-gitlab> gitlab # [ 226.092187] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.972Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3969a755c7d9969293208b5c","created_at":"2026-06-29T21:57:13.710Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.711Z","job_size_bytes":66,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3969a755c7d9969293208b5c: done: 0.019063 sec","job_status":"done","queue_duration_s":0.241529,"scheduling_latency_s":0.241554,"redis_calls":7,"redis_duration_s":0.001966,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001526,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019063,"completed_at":"2026-06-29T21:57:13.972Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001126,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2258vm-test-run-nixbot-gitlab> gitlab # [ 226.109094] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.973Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"68d62138203ae8073a1a49dd","created_at":"2026-06-29T21:57:13.712Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.714Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-68d62138203ae8073a1a49dd: start","job_status":"start","queue_duration_s":0.259403,"scheduling_latency_s":0.259431}2259vm-test-run-nixbot-gitlab> gitlab # [ 226.135119] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.998Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"68d62138203ae8073a1a49dd","created_at":"2026-06-29T21:57:13.712Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.714Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-68d62138203ae8073a1a49dd: done: 0.02427 sec","job_status":"done","queue_duration_s":0.259403,"scheduling_latency_s":0.259431,"redis_calls":7,"redis_duration_s":0.0016330000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02427,"completed_at":"2026-06-29T21:57:13.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002692,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2260vm-test-run-nixbot-gitlab> gitlab # [ 226.235169] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:13.999Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"94b74f091266bcf1c47380fe","created_at":"2026-06-29T21:57:13.749Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.751Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-94b74f091266bcf1c47380fe: start","job_status":"start","queue_duration_s":0.247754,"scheduling_latency_s":0.247777}2261vm-test-run-nixbot-gitlab> gitlab # [ 226.236372] gitaly[1387]: time="2026-06-29T21:57:13.744Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:43.743" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:13.743" grpc.time_ms=0.664 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2262vm-test-run-nixbot-gitlab> gitlab # [ 226.245903] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:14.018Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"94b74f091266bcf1c47380fe","created_at":"2026-06-29T21:57:13.749Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.751Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-94b74f091266bcf1c47380fe: done: 0.019105 sec","job_status":"done","queue_duration_s":0.247754,"scheduling_latency_s":0.247777,"redis_calls":7,"redis_duration_s":0.00165,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019105,"completed_at":"2026-06-29T21:57:14.018Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2263vm-test-run-nixbot-gitlab> gitlab # [ 226.377658] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:14.021Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"fd02098ab43adf804243f588","created_at":"2026-06-29T21:57:13.752Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.753Z","job_size_bytes":6,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-fd02098ab43adf804243f588: start","job_status":"start","queue_duration_s":0.267278,"scheduling_latency_s":0.267317}2264vm-test-run-nixbot-gitlab> gitlab # [ 226.378727] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:14.036Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"fd02098ab43adf804243f588","created_at":"2026-06-29T21:57:13.752Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.753Z","job_size_bytes":6,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-fd02098ab43adf804243f588: done: 0.016035 sec","job_status":"done","queue_duration_s":0.267278,"scheduling_latency_s":0.267317,"redis_calls":7,"redis_duration_s":0.001601,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016035,"completed_at":"2026-06-29T21:57:14.036Z","load_balancing_strategy":"primary","db_duration_s":0.000597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2265vm-test-run-nixbot-gitlab> gitlab # [ 226.403512] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:14.038Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"3f22b84e9b2e8044844712f8","created_at":"2026-06-29T21:57:13.754Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.757Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3f22b84e9b2e8044844712f8: start","job_status":"start","queue_duration_s":0.28048,"scheduling_latency_s":0.280535}2266vm-test-run-nixbot-gitlab> gitlab # [ 226.428114] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:14.063Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"3f22b84e9b2e8044844712f8","created_at":"2026-06-29T21:57:13.754Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.757Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3f22b84e9b2e8044844712f8: done: 0.025649 sec","job_status":"done","queue_duration_s":0.28048,"scheduling_latency_s":0.280535,"redis_calls":10,"redis_duration_s":0.002262,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000334679,"duration_s":0.025649,"completed_at":"2026-06-29T21:57:14.063Z","load_balancing_strategy":"primary","db_duration_s":0.004537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2267vm-test-run-nixbot-gitlab> gitlab # [ 226.520980] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:14.065Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"9d475b652c49407b2f4b091d","created_at":"2026-06-29T21:57:13.794Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.795Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9d475b652c49407b2f4b091d: start","job_status":"start","queue_duration_s":0.269484,"scheduling_latency_s":0.269504}2268vm-test-run-nixbot-gitlab> gitlab # [ 226.522371] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:14.174Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"9d475b652c49407b2f4b091d","created_at":"2026-06-29T21:57:13.794Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-0d4b382913804b89","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:13.795Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9d475b652c49407b2f4b091d: done: 0.109312 sec","job_status":"done","queue_duration_s":0.269484,"scheduling_latency_s":0.269504,"gitaly_calls":3,"gitaly_duration_s":0.009714,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007716,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001098,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001822,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000292,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001648,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000287,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000429,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074451,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.089387059,"duration_s":0.109312,"completed_at":"2026-06-29T21:57:14.174Z","load_balancing_strategy":"primary","db_duration_s":0.024016,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2269vm-test-run-nixbot-gitlab> gitlab # [ 226.662359] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:14.176Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"2091a2b44916c92afc425a33","created_at":"2026-06-29T21:57:14.110Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-07fe526533f148f3","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:14.112Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2091a2b44916c92afc425a33: start","job_status":"start","queue_duration_s":0.064658,"scheduling_latency_s":0.064681}2270vm-test-run-nixbot-gitlab> gitlab # [ 226.663868] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:14.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"2091a2b44916c92afc425a33","created_at":"2026-06-29T21:57:14.110Z","trace_propagation_headers":{"sentry-trace":"b34f5b8e8b5f44548d8b0eea7bb73218-07fe526533f148f3","baggage":"sentry-trace_id=b34f5b8e8b5f44548d8b0eea7bb73218,sentry-release=Unknown"},"correlation_id":"01KWAP5SX18N1YEKKRBQAF7HQN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:14.112Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2091a2b44916c92afc425a33: done: 0.166619 sec","job_status":"done","queue_duration_s":0.064658,"scheduling_latency_s":0.064681,"redis_calls":7,"redis_duration_s":0.019599,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001977,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.017622,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.166619,"completed_at":"2026-06-29T21:57:14.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2271vm-test-run-nixbot-gitlab> gitlab # [ 226.691374] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.381Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"68dc29647a45b1458a4b042f","created_at":"2026-06-29T21:57:17.376Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-692640506801471a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.378Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-68dc29647a45b1458a4b042f: start","job_status":"start","queue_duration_s":0.002801,"scheduling_latency_s":0.002841}2272vm-test-run-nixbot-gitlab> gitlab # [ 226.716951] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.725Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"68dc29647a45b1458a4b042f","created_at":"2026-06-29T21:57:17.376Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-692640506801471a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.378Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-68dc29647a45b1458a4b042f: done: 0.343431 sec","job_status":"done","queue_duration_s":0.002801,"scheduling_latency_s":0.002841,"gitaly_calls":4,"gitaly_duration_s":0.142172,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.030502,"redis_read_bytes":173,"redis_write_bytes":20930,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002277,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002192,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.007347,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13383,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.008583,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00031,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002334,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.005182,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.024,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121466,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.314587748,"duration_s":0.343431,"completed_at":"2026-06-29T21:57:17.725Z","load_balancing_strategy":"primary","db_duration_s":0.048006,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2273vm-test-run-nixbot-gitlab> gitlab # [ 226.806106] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.728Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8aa53017fdc4cf5a4731b81b","created_at":"2026-06-29T21:57:17.446Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.448Z","job_size_bytes":23,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8aa53017fdc4cf5a4731b81b: start","job_status":"start","queue_duration_s":0.279837,"scheduling_latency_s":0.279879}2274vm-test-run-nixbot-gitlab> gitlab # [ 226.822470] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.777Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8aa53017fdc4cf5a4731b81b","created_at":"2026-06-29T21:57:17.446Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.448Z","job_size_bytes":23,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8aa53017fdc4cf5a4731b81b: done: 0.049098 sec","job_status":"done","queue_duration_s":0.279837,"scheduling_latency_s":0.279879,"redis_calls":7,"redis_duration_s":0.006457,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00588,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000577,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049098,"completed_at":"2026-06-29T21:57:17.777Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010673,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2275vm-test-run-nixbot-gitlab> gitlab # [ 226.947629] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.778Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a6f092031676e4b7908d515f","created_at":"2026-06-29T21:57:17.592Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.600Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a6f092031676e4b7908d515f: start","job_status":"start","queue_duration_s":0.178107,"scheduling_latency_s":0.178131}2276vm-test-run-nixbot-gitlab> gitlab # [ 226.949200] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.854Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a6f092031676e4b7908d515f","created_at":"2026-06-29T21:57:17.592Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.600Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a6f092031676e4b7908d515f: done: 0.075751 sec","job_status":"done","queue_duration_s":0.178107,"scheduling_latency_s":0.178131,"redis_calls":7,"redis_duration_s":0.052513,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004765,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.047748,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075751,"completed_at":"2026-06-29T21:57:17.854Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2277vm-test-run-nixbot-gitlab> gitlab # [ 226.983280] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.868Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c983902cdcc161b361bd3d85","created_at":"2026-06-29T21:57:17.612Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.617Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c983902cdcc161b361bd3d85: start","job_status":"start","queue_duration_s":0.25133,"scheduling_latency_s":0.251381}2278vm-test-run-nixbot-gitlab> gitlab # [ 227.008493] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.907Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c983902cdcc161b361bd3d85","created_at":"2026-06-29T21:57:17.612Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.617Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c983902cdcc161b361bd3d85: done: 0.039025 sec","job_status":"done","queue_duration_s":0.25133,"scheduling_latency_s":0.251381,"redis_calls":7,"redis_duration_s":0.004395,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039025,"completed_at":"2026-06-29T21:57:17.907Z","load_balancing_strategy":"primary","db_duration_s":0.005599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279vm-test-run-nixbot-gitlab> gitlab # [ 227.091137] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.909Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"a9e98d16460e18bd47771abb","created_at":"2026-06-29T21:57:17.626Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.627Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a9e98d16460e18bd47771abb: start","job_status":"start","queue_duration_s":0.281318,"scheduling_latency_s":0.281341}2280vm-test-run-nixbot-gitlab> gitlab # [ 227.097929] gitaly[1387]: time="2026-06-29T21:57:13.780Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:43.777" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:13.779" grpc.time_ms=0.667 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2281vm-test-run-nixbot-gitlab> gitlab # [ 227.117295] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.932Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"a9e98d16460e18bd47771abb","created_at":"2026-06-29T21:57:17.626Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.627Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a9e98d16460e18bd47771abb: done: 0.023805 sec","job_status":"done","queue_duration_s":0.281318,"scheduling_latency_s":0.281341,"redis_calls":10,"redis_duration_s":0.002399,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000652,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023805,"completed_at":"2026-06-29T21:57:17.932Z","load_balancing_strategy":"primary","db_duration_s":0.002427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2282vm-test-run-nixbot-gitlab> gitlab # [ 227.232411] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.934Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"21f67235cebcfe730cf84a9f","created_at":"2026-06-29T21:57:17.628Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.629Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-21f67235cebcfe730cf84a9f: start","job_status":"start","queue_duration_s":0.304752,"scheduling_latency_s":0.304774}2283vm-test-run-nixbot-gitlab> gitlab # [ 227.233868] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.953Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"21f67235cebcfe730cf84a9f","created_at":"2026-06-29T21:57:17.628Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.629Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-21f67235cebcfe730cf84a9f: done: 0.018665 sec","job_status":"done","queue_duration_s":0.304752,"scheduling_latency_s":0.304774,"redis_calls":7,"redis_duration_s":0.0017460000000000002,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001224,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018665,"completed_at":"2026-06-29T21:57:17.953Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2284vm-test-run-nixbot-gitlab> gitlab # [ 227.278848] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.954Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8acb241c2ea05de4790e5c5b","created_at":"2026-06-29T21:57:17.630Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.633Z","job_size_bytes":19,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8acb241c2ea05de4790e5c5b: start","job_status":"start","queue_duration_s":0.3215,"scheduling_latency_s":0.321524}2285vm-test-run-nixbot-gitlab> gitlab # [ 227.304407] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.974Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8acb241c2ea05de4790e5c5b","created_at":"2026-06-29T21:57:17.630Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.633Z","job_size_bytes":19,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8acb241c2ea05de4790e5c5b: done: 0.020439 sec","job_status":"done","queue_duration_s":0.3215,"scheduling_latency_s":0.321524,"redis_calls":8,"redis_duration_s":0.002399,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001973,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020439,"completed_at":"2026-06-29T21:57:17.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002269,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2286vm-test-run-nixbot-gitlab> gitlab # [ 227.376175] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.976Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0e5ab7c918e09915f0a8d3cc","created_at":"2026-06-29T21:57:17.634Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.635Z","job_size_bytes":66,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0e5ab7c918e09915f0a8d3cc: start","job_status":"start","queue_duration_s":0.34046,"scheduling_latency_s":0.340481}2287vm-test-run-nixbot-gitlab> gitlab # [ 227.397547] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.991Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0e5ab7c918e09915f0a8d3cc","created_at":"2026-06-29T21:57:17.634Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.635Z","job_size_bytes":66,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0e5ab7c918e09915f0a8d3cc: done: 0.015574 sec","job_status":"done","queue_duration_s":0.34046,"scheduling_latency_s":0.340481,"redis_calls":7,"redis_duration_s":0.00163,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001181,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015574,"completed_at":"2026-06-29T21:57:17.991Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2288vm-test-run-nixbot-gitlab> gitlab # [ 227.520239] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:17.993Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"5c54c6e5bd1cb7b7f5bb22fe","created_at":"2026-06-29T21:57:17.637Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.640Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5c54c6e5bd1cb7b7f5bb22fe: start","job_status":"start","queue_duration_s":0.352818,"scheduling_latency_s":0.35284}2289vm-test-run-nixbot-gitlab> gitlab # [ 227.521752] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:18.017Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"5c54c6e5bd1cb7b7f5bb22fe","created_at":"2026-06-29T21:57:17.637Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.640Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5c54c6e5bd1cb7b7f5bb22fe: done: 0.023898 sec","job_status":"done","queue_duration_s":0.352818,"scheduling_latency_s":0.35284,"redis_calls":7,"redis_duration_s":0.001748,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001263,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000485,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023898,"completed_at":"2026-06-29T21:57:18.017Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2290vm-test-run-nixbot-gitlab> gitlab # [ 227.560230] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:18.019Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"e1a1317896531fd4c15fbeb7","created_at":"2026-06-29T21:57:17.674Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.675Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e1a1317896531fd4c15fbeb7: start","job_status":"start","queue_duration_s":0.343261,"scheduling_latency_s":0.343283}2291vm-test-run-nixbot-gitlab> gitlab # [ 227.586895] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:18.038Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"e1a1317896531fd4c15fbeb7","created_at":"2026-06-29T21:57:17.674Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.675Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e1a1317896531fd4c15fbeb7: done: 0.019452 sec","job_status":"done","queue_duration_s":0.343261,"scheduling_latency_s":0.343283,"redis_calls":7,"redis_duration_s":0.001596,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019452,"completed_at":"2026-06-29T21:57:18.038Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2292vm-test-run-nixbot-gitlab> gitlab # [ 227.665994] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:18.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0b0de4620e028eca37269e3d","created_at":"2026-06-29T21:57:17.676Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.677Z","job_size_bytes":6,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0b0de4620e028eca37269e3d: start","job_status":"start","queue_duration_s":0.362143,"scheduling_latency_s":0.362166}2293vm-test-run-nixbot-gitlab> gitlab # [ 227.678845] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:18.053Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0b0de4620e028eca37269e3d","created_at":"2026-06-29T21:57:17.676Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.677Z","job_size_bytes":6,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0b0de4620e028eca37269e3d: done: 0.013837 sec","job_status":"done","queue_duration_s":0.362143,"scheduling_latency_s":0.362166,"redis_calls":7,"redis_duration_s":0.0016979999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000512,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013837,"completed_at":"2026-06-29T21:57:18.053Z","load_balancing_strategy":"primary","db_duration_s":0.000849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2294vm-test-run-nixbot-gitlab> gitlab # [ 227.748113] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:18.055Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"58d91742742273b85d85ccc5","created_at":"2026-06-29T21:57:17.678Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.679Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-58d91742742273b85d85ccc5: start","job_status":"start","queue_duration_s":0.37566,"scheduling_latency_s":0.375681}2295vm-test-run-nixbot-gitlab> gitlab # [ 227.769252] sidekiq-cluster[2173]: {"severity":"INFO","time":"2026-06-29T21:57:18.076Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"58d91742742273b85d85ccc5","created_at":"2026-06-29T21:57:17.678Z","trace_propagation_headers":{"sentry-trace":"d43d5e17b47a4baab47374351843c2c9-5a65ad7575124a7a","baggage":"sentry-trace_id=d43d5e17b47a4baab47374351843c2c9,sentry-release=Unknown"},"correlation_id":"01KWAP5ZPJJ5PMRBQJCZSSEQ0S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-29T21:57:17.679Z","job_size_bytes":3,"pid":2173,"sidekiq_tid":"maad","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-58d91742742273b85d85ccc5: done: 0.020965 sec","job_status":"done","queue_duration_s":0.37566,"scheduling_latency_s":0.375681,"redis_calls":7,"redis_duration_s":0.00597,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004807,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020965,"completed_at":"2026-06-29T21:57:18.076Z","load_balancing_strategy":"primary","db_duration_s":0.001962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2296vm-test-run-nixbot-gitlab> gitlab # [ 227.861601] gitaly[1387]: time="2026-06-29T21:57:13.791Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5WCR91FT9YXAB16A6VNA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:43.789" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:13.790" grpc.time_ms=0.681 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2297vm-test-run-nixbot-gitlab> gitlab # [ 227.862950] gitaly[1387]: time="2026-06-29T21:57:14.122Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T21:57:44.120" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:14.121" grpc.time_ms=0.935 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2298vm-test-run-nixbot-gitlab> gitlab # [ 227.874229] gitaly[1387]: time="2026-06-29T21:57:14.143Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:44.142" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:14.142" grpc.time_ms=0.692 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2299vm-test-run-nixbot-gitlab> gitlab # [ 227.950155] gitaly[1387]: time="2026-06-29T21:57:14.164Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5SX18N1YEKKRBQAF7HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:44.163" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:14.163" grpc.time_ms=0.599 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2300vm-test-run-nixbot-gitlab> gitlab # [ 227.951435] gitaly[1387]: time="2026-06-29T21:57:14.927Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5XFRSE1D5AVM63ZJJGCF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:44.024" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:14.925" grpc.time_ms=1.189 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2301vm-test-run-nixbot-gitlab> gitlab # [ 227.952268] gitaly[1387]: time="2026-06-29T21:57:16.082Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5YKPDF5X2A0CFRRC3KBG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:46.081" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:16.081" grpc.time_ms=0.906 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2302vm-test-run-nixbot-gitlab> gitlab # [ 227.953174] gitaly[1387]: time="2026-06-29T21:57:17.321Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:47.320" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:17.320" grpc.time_ms=0.698 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2303vm-test-run-nixbot-gitlab> gitlab # [ 227.972306] gitaly[1387]: time="2026-06-29T21:57:17.355Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T21:57:27.216" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:17.346" grpc.time_ms=7.878 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2304vm-test-run-nixbot-gitlab> gitlab # [ 227.993747] gitaly[1387]: time="2026-06-29T21:57:17.481Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:17.481" grpc.time_ms=0.386 pid=1387 span.kind=server system=grpc2305vm-test-run-nixbot-gitlab> gitlab # [ 228.008987] gitaly[1387]: time="2026-06-29T21:57:17.504Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:17.504" grpc.time_ms=0.463 pid=1387 span.kind=server system=grpc2306vm-test-run-nixbot-gitlab> gitlab # [ 228.075532] gitaly[1387]: time="2026-06-29T21:57:17.541Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T21:57:17.540" grpc.time_ms=0.433 pid=1387 span.kind=server system=grpc2307vm-test-run-nixbot-gitlab> gitlab # [ 228.077187] gitaly[1387]: time="2026-06-29T21:57:17.551Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:47.453" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:17.453" operations="[repack_refs]" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2308vm-test-run-nixbot-gitlab> gitlab # [ 228.078201] gitaly[1387]: time="2026-06-29T21:57:17.552Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:47.453" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:17.453" pid=1387 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-06-29 21:56:54.528377886 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2309vm-test-run-nixbot-gitlab> gitlab # [ 228.079089] gitaly[1387]: time="2026-06-29T21:57:17.583Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S diskcache=02f2f31f-9ce3-43f2-9b1f-0bdb9638c5f4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:47.453" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:17.453" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2310vm-test-run-nixbot-gitlab> gitlab # [ 228.087257] gitaly[1387]: time="2026-06-29T21:57:17.583Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=98 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7649 command.oublock=0 command.real_time_ms=99 command.spawn_token_fork_ms=2 command.system_time_ms=70 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:47.453" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:17.453" grpc.time_ms=129.576 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2311vm-test-run-nixbot-gitlab> gitlab # [ 228.108705] gitaly[1387]: time="2026-06-29T21:57:17.595Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:47.453" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:17.453" pid=1387 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2312vm-test-run-nixbot-gitlab> gitlab # [ 228.129501] gitaly[1387]: time="2026-06-29T21:57:17.596Z" level=info msg="optimized repository" clean_stale_data_time=21.337908ms clean_work_trees_time="94.984µs" commit_graph_time="8.381µs" correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.661µs" pid=1387 prune_time="7.822µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.66µs" storage_name=default system=housekeeping total_time=21.676219ms2313vm-test-run-nixbot-gitlab> gitlab # [ 228.139469] gitaly[1387]: time="2026-06-29T21:57:17.596Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T21:57:47.453" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T21:57:17.453" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2314vm-test-run-nixbot-gitlab> gitlab # [ 228.219289] gitaly[1387]: time="2026-06-29T21:57:17.647Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T21:57:47.646" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:17.646" grpc.time_ms=1.112 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2315vm-test-run-nixbot-gitlab> gitlab # [ 228.220420] gitaly[1387]: time="2026-06-29T21:57:17.671Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:47.669" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:17.670" grpc.time_ms=0.938 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2316vm-test-run-nixbot-gitlab> gitlab # [ 228.221234] gitaly[1387]: time="2026-06-29T21:57:17.711Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP5ZPJJ5PMRBQJCZSSEQ0S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:47.710" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:17.711" grpc.time_ms=0.53 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2317vm-test-run-nixbot-gitlab> gitlab # [ 228.222127] gitaly[1387]: time="2026-06-29T21:57:17.795Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWAP6096BKQ93MSM85CGE51H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:17.782" grpc.time_ms=13.231 pid=1387 remote_ip=127.0.0.1 span.kind=server system=grpc2318vm-test-run-nixbot-gitlab> gitlab # [ 228.234233] gitaly[1387]: time="2026-06-29T21:57:18.105Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWAP60B830C9PBC234D2M3CQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:18.094" pid=1387 remote_ip=127.0.0.1 request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2319vm-test-run-nixbot-gitlab> gitlab # [ 228.250276] gitaly[1387]: time="2026-06-29T21:57:18.106Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP60B830C9PBC234D2M3CQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:18.094" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip=127.0.0.1 span.kind=server system=grpc2320vm-test-run-nixbot-gitlab> gitlab # [ 228.272137] gitaly[1387]: time="2026-06-29T21:57:18.106Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP60B830C9PBC234D2M3CQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T21:57:18.094" grpc.time_ms=11.807 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip=127.0.0.1 span.kind=server system=grpc2321vm-test-run-nixbot-gitlab> gitlab # [ 228.415588] gitaly[1387]: time="2026-06-29T21:57:21.407Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWAP63TJ8TMYAPZTWKDBYX1A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T21:57:51.405" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T21:57:21.406" grpc.time_ms=1.118 pid=1387 remote_ip=127.0.0.1 span.kind=server system=grpc username=root2322vm-test-run-nixbot-gitlab> gitlab # [ 228.437533] gitlab-workhorse[1305]: gitlab 127.0.0.1 - - [2026/06/29:21:57:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 662323vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8b5212701fe6acf76e74f205791f1e298c878a79/statuses, in 4.37 seconds)2324vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T21:57:05.776Z', 'started_at': None, 'finished_at': '2026-06-29T21:57:13.494Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '8b5212701fe6acf76e74f205791f1e298c878a79', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-06-29T21:57:13.627Z', 'started_at': None, 'finished_at': '2026-06-29T21:57:17.369Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2325vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 15.91 seconds)2326vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 264.21 seconds)2327vm-test-run-nixbot-gitlab> test script finished in 264.25s2328vm-test-run-nixbot-gitlab> cleanup2329vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2330vm-test-run-nixbot-gitlab> gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2331vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.35 seconds)23322333post-build step Upload coverage to codecov: ok2334Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab